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FOREWORD 


This  work  was  funded  as  part  of  the  Advanced  Development  project  entitled  Low 
Cost  Microcomputer  Training  Systems  (Program  Element  Number  63720N,  Work  Unit 
Number  Z1772-ET002).  The  project  was  the  result  of  an  operational  requirement 
promulgated  by  the  Chief  of  Naval  Operations  (OP-987H,  OP-01B7). 

Two  surveys  of  Navy  technical  training  course  managers  and  instructors  were 
conducted  to  analyze  Navy  training  and  instructional  practices.  This  report  presents  the 
instructional  delivery  and  course  management  techniques  currently  used  in  the  courses 
surveyed  and  the  assessment  of  surveyed  personnel  concerning  the  appropriateness  and 
acceptability  of  microcomputer  support  in  these  areas. 

The  results  of  the  other  survey,  which  examined  the  learning  objectives  of  245  Navy 
technical  training  courses,  will  be  published  when  available.  The  results  of  the  present 
study  are  primarily  intended  for  the  Department  of  the  Navy  training  community. 


B.  E.  BACON 
Captain,  U.S.  Navy 
Commanding  Officer 


J.  S.  McMICHAEL 
Technical  Director 
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SUMMARY 


Background  and  Problem 

There  is  a  high  degree  of  interest  in  using  microcomputers  in  military  training  to 
improve  instruction  or  to  supplement  limited  instructor  resources.  An  important  issue  for 
Navy  training  is  to  determine  the  requirements  for  microcomputer  support  in  both  the 
direct  training  of  students  and  in  the  general  management  of  training.  One  approach  to 
this  issue  is  to  gain  relief  through  the  use  of  computer-based  instruction  (CBI)  to  augment 
the  instructional  staff  in  the  delivery  and  management  of  instruction. 

Objective 

This  survey  was  conducted  to  determine  the  instructional  delivery  and  course 
management  techniques  currently  used  in  Navy  technical  courses  and  to  assess  the 
suitability  of  microcomputer  support  in  such  courses. 

Approach 

On-site  structured  interviews  were  conducted  with  senior  instructors  or  course 
managers  of  135  Navy  courses.  The  courses  had  an  annual  throughput  of  at  least  400 
students  or  had  been  identified  as  having  critical  shortfalls.  The  data  were  analyzed  in 
terms  of  the  type  of  school  (A-school  versus  advanced  C-<Sc-F-schools),  occupational  group 
(electrical,  mechanical,  clerical,  operator,  and  team),  and  course  duration. 

Findings 

The  percentage  of  users  and  average  percent  of  time  the  surveyed  courses  devoted  to 
different  instructional  methods  fell  into  three  groups:  76  percent  usage  (lecture  and  lab) 
for  40  percent  of  the  time;  56  to  62  percent  (tests,  films  or  videotape,  and  demonstration) 
for  5  to  10  percent  of  the  time;  22  to  35  percent  (self-study,  discussion,  tutoring,  drill, 
and  practice)  for  7  to  30  percent  of  the  time.  An  analysis  of  the  test  methods  currently 
employed  showed  the  greatest  use  of  computers  for  general  administrative  test  scoring, 
analysis,  and  recording.  Course  managers  frequently  cited  training  objectives  calling  for 
simulation  (63%)  and  drill  and  practice  (54%).  This  result  coincided  with  a  companion 
analysis  of  actual  training  objectives  that  found  the  use  of  procedures  and  remembering  of 
facts  to  be  quite  frequent.  Training  objectives  for  computer  use  were  reported  by  18 
percent  of  the  sample,  while  26  percent  reported  that  the  student  would  use  a  computer 
on  the  job  after  completing  the  course. 

The  first  ranked  priority  for  computer  support  was  for  course  managers  (97%). 
General  administrative  or  clerical  computer  support  in  the  area  of  registering,  scheduling, 
tracking,  score  recording,  and  general  record  keeping  were  cited  as  desirable  items  to 
add.  The  next  ranked  area  for  computer  support  included  using  CBI.  In  assessing  future 
interest,  about  27  percent  of  the  courses  nominated  at  least  one  module  as  suitable  for 
CBI.  Currently  about  12.6  percent  of  all  sampled  courses  used  some  form  of  CBI  (20%  in 
A-schools  and  5.6%  in  C-&-F -schools).  Not  surprisingly,  most  of  these  were  in  electrical 
related  schools  (30%). 

Special  problems  in  curriculum  stability  (39%),  inadequate  learning  objectives  (39%), 
students'  entering  skills  (33%),  and  students'  abilities  in  math  (35%),  and  reading  (46%) 
were  reported.  A  severe  student  "wait  time"  for  access  to  laboratory  equipment  was 
reported  by  13  percent  of  the  course  managers,  with  A-schools  reporting  a  more  severe 


problem  (20.3%)  than  C-ic-F -schools  (7%).  About  14  percent  of  the  students  were 
reported  to  not  reach  criterion  on  the  first  attempt  of  a  module  test,  with  36  percent  of 
the  managers  wanting  to  track  such  test  attempts  and  another  21  percent  being 
dissatisfied  with  their  current  tracking  process.  Overall,  the  managers  estimated  that 
fast  and  slow  students  differed  by  about  8  days  in  completing  courses. 

Discussion  and  Conclusions 


The  lowest  common  denominator  to  all  of  the  courses  surveyed  was  the  need  for  some 
general  administrative  and  clerical  computer  support  in  the  area  of  registering,  schedul- 
tracking,  score  recording,  and  general  record  keeping.  These  training  management 
functions  may  become  more  complicated  and  less  centralized  in  view  of  recent  initiatives 
to  move  training  from  formal  classrooms  to  shipboard  or  on-the-job  training.  While 
improvements  resulting  from  easing  clerical  or  administrative  tedium  are  easy  to  foresee, 
the  value  of  CBI  applications  in  delivering  instruction  requires  scrutiny.  Rather  than 
computerizing  entire  curricula,  a  more  rational  path  is  to  identify  selected  CBI  applica¬ 
tions  that  do  offer  an  improvement.  Some  practical  reasons  for  using  CBI  were  (1) 
offering  learning  capabilities  not  possible  with  conventional  methods,  (2)  reducing  costs 
compared  to  those  of  high  fidelity  trainers,  (3)  supplementing  instructor  resources,  (4) 
standardizing  instruction  over  many  sites,  (5)  on-site  individual  training,  and  (6)  possible 
reductions  in  training  time.  Severe  student  wait  times  for  access  to  laboratory  equipment 
could  be  reduced  with  less  expensive  CBI  systems  employed  as  "ancillary  trainers"  to 
teach  related  information  or  component  skills  pertinent  to  the  more  expensive  trainers  for 
which  access  is  limited. 

Recommendations 


1.  Computer  support  should  be  provided  for  many  administrative  and  clerical 
functions  such  as  scheduling,  student  tracking,  and  general  record  keeping.  Student 
testing  programs  with  a  stable  high  volume  would  benefit  from  computerized  test 
administration,  scoring,  recording,  and  tracking  functions. 

2.  With  recent  initiatives  to  move  training  out  of  shore-based  schools,  training 
management,  student  record  keeping,  and  certification  testing  will  be  more  diffuse  and 
harder  to  manage  in  the  future.  Therefore,  stand-alone  computer  tools  on  standard 
microcomputers  should  be  developed  for  shipboard  personnel  to  use  in  managing  training. 

3.  CBI  should  be  used  for  a  number  of  specific  applications.  Using  ancillary 
trainers  for  some  laboratory  situations  would  allow  better  utilization  of  student  and 
instructor  time  when  limited  access  to  high  cost  trainers  causes  excessive  waiting.  Many 
students'  entering  skills  (e.g.,  reading,  mathematics,  technical  vocabulary)  should  be 
supplemented  with  commonly  available  CBI.  Learning  objectives  involving  drill  and 
practice,  simulation,  remembering  facts,  and  use  of  procedural  steps  occur  frequently  and 
are  particularly  amenable  to  CBI. 

4.  Continuing  work  should  be  supported  to  develop  guidelines  as  to  when  CBI  is 
appropriate  and  to  develop  aids  for  CBI  authors  so  that  they  can  develop  quality  CBI  for 
their  students. 
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INTRODUCTION 


Problem  and  Background 

The  current  high  interest  in  using  microcomputers  in  military  training  reflects  the 
general  increase  in  use  of  computers  in  our  society  and  their  use  in  education  to  improve 
instruction  or  supplement  limited  instructor  resources.  An  important  problem  for  Navy 
training  personnel  is  to  determine  what  requirements  exist  for  microcomputer  support  in 
both  the  direct  training  of  students  and  in  the  general  management  of  training.  To  do  this 
requires  a  better  understanding  of  the  kinds  of  training  that  are  most  amenable  to 
computerization,  the  software  available  to  meet  these  requirements,  and  the  require¬ 
ments  that  available  software  does  not  meet.  A  variety  of  individual  evaluations  indicate 
that  various  forms  of  computer-based  instruction  (CBI)  can  make  good  gains  in  efficiency 
and  effectiveness  for  some  kinds  of  training  (Bangert-Drowns,  Kulik,  &  Kulik,  1985; 
Blaiwes  &  Regan,  1986;  Fletcher  &  Rockway,  1986;  Kearsley,  1983;  Orlansky  &  String, 
1981).  However,  there  is  no  general  strategy  to  guide  investments  in  automation  in 
training  or  to  project  system  requirements  for  the  future.  Accordingly,  the  delivery  and 
management  techniques  used  in  Navy  technical  schools  as  well  as  appropriateness  and 
acceptability  of  microcomputer  support  in  these  areas  need  to  be  assessed. 

The  increased  complexity  of  current  and  projected  Navy  equipment  places  additional 
demands  on  the  Navy  training  establishment  to  provide  the  skilled  operators  and 
maintainers  the  Fleet  needs  to  achieve  operational  readiness.  This  increases  the  need  for 
senior  technicians  and  operators  to  staff  the  schools.  At  the  same  time,  the  growth  of  the 
Navy  has  increased  the  Fleet  needs  for  experienced  personnel  to  provide  supervision  and 
leadership  for  the  new  units.  School  staffs  may  not  grow  to  match  the  increased  demand, 
but  will  have  to  train  more  students  with  equal  or  reduced  staffing.  Recent  initiatives  to 
move  training  from  shore-based  schools  to  ships  will  make  training  and  training  manage¬ 
ment  even  less  centralized  and  harder  to  manage  in  the  future. 

Computer-aided  instruction  (CAI)  and  computer-managed  instruction  (CMI)  may  be 
used  to  augment  the  instructional  staff  in  the  delivery  and  management  of  instruction.  In 
CAI,  computers  actually  deliver  the  instruction.  Many  interactive  CAI  sequences  present 
new  information  to  students  and  then  query  their  understanding.  Tailored  branching 
directs  students  who  answer  incorrectly  to  remedial  or  explanatory  tracts.  Other 
situations  may  involve  simulation  of  principles  not  easily  conveyed  in  conventional  book 
form  or  practice  on  procedural  steps  such  as  the  operation  of  equipment.  In  CMI, 
computers  manage  instruction  rather  than  deliver  the  actual  instruction.  Computers  are 
used  to  automate  many  clerical  aspects  of  record  keeping;  for  example,  scheduling  of 
resources,  students,  and  instructors;  preparing  reports;  and  keeping  track  of  student 
information  such  as  class  rosters,  test  scores  (perhaps  scoring  tests  and  providing 
feedback),  and  student  progress  over  course  modules.  Often,  the  general  term  CBI  is  used 
to  represent  either  CAI  or  CMI  or  both. 

Neither  CAI  nor  CMI  should  be  viewed  as  a  panacea  by  the  naval  education  and 
training  community.  Not  all  instruction  is  appropriate  for  CAI  (Montague  &  Wulfeck, 
1984).  Selection  guidelines  must  be  developed  in  order  to  identify  the  Navy  schools  that 
are  suitable  for  CAI  and/or  computerized  course  management. 

Purpose 

This  survey  was  conducted  to  determine  the  instructional  delivery  and  course 
management  techniques  currently  used  in  Navy  technical  courses  and  to  assess  the 
suitability  of  microcomputer  support  in  such  courses. 
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APPROACH 


Sample  Selection 

The  Navy  maintains  over  4800  courses  of  instruction,  most  of  which  graduate  less 
than  100  students  a  year.  Since  converting  lecture-based  instruction  to  some  form  of  CBI 
requires  substantial  resources  (Van  Kekerix,  Wulfeck,  &  Montague,  1982),  many  of  these 
schools  lack  the  manpower  to  implement  CBI.  Therefore,  only  the  managers  and  senior 
instructors  of  courses  with  an  annual  throughput  of  400  or  more  students  (i.e.,  large 
enough  to  support  a  transition  effort)  were  interviewed.  With  large  student  throughput, 
the  costs  of  computerization  can  be  amortized  over  a  large  number  of  students  making  for 
greater  cost  efficiencies.  However,  Van  Kekerix,  Wulfeck,  and  Montague  (1982)  showed 
that  CBI  can  avoid  substantial  costs  even  for  courses  with  very  few  students. 

Courses  with  an  annual  throughput  of  400  students  were  extracted  from  the  Navy 
Integrated  Training  Resources  and  Administration  System  (NITRAS)  data  base  maintained 
by  the  Naval  Education  and  Training  Command  (NAVEDTRACOM).  Submarine  courses 
were  not  included  in  this  initial  selection  at  the  request  of  the  Chief  of  Naval  Education 
and  Training  (CNET).  The  initial  selection  was  augmented  with  courses  of  shorter 
duration  for  the  32  technical  ratings  Koehler  (1982)  identified  as  facing  critical  personnel 
shortfalls  through  FY87.  These  "critical"  courses  especially  need  assistance  to  enhance 
the  performance  of  schoolhouse  personnel.  The  initial  selection  process  resulted  in  a  total 
of  340  candidate  courses  for  the  survey. 

The  340  candidate  courses  were  reduced  by:  (1)  the  Chief  of  Naval  Technical 
Training  (CNTT)  limiting  the  number  of  courses  that  could  be  surveyed  at  a  single 
command,  (2)  dropping  courses  being  revised  or  discontinued,  (3)  financial  constraints  that 
limited  travel  to  sites  with  at  least  five  courses,  and  (4)  four  course  managers  who  each 
addressed  all  of  the  courses  they  controlled  in  a  single  interview. 

The  final  survey  consisted  of  135  course  managers  and  senior  instructors,  who  were 
interviewed  from  1983  through  1984. 

Survey  Questionnaire 

The  survey  questionnaire  was  developed  in  conjunction  with  the  NAVEDTRACOM 
Training  Analysis  and  Evaluation  Group  (TAEG)1  to  obtain  information  related  to  the 
suitability,  desirability,  and  acceptability  of  CAI  and  CMI.  The  questionnaire  (Appendix 
A)  was  administered  as  an  on-site  structured  interview  with  course  managers  or  senior 
instructors.  The  questions  formed  clusters  focusing  on  specific  areas  of  instructional 
formats,  instructional  content,  student  problems,  training  objectives,  testing,  managerial 
functions,  and  the  amount  of  time  spent  in  the  various  instructional  activities.  Other 
details  of  the  questionnaire  and  the  rationale  for  its  development  are  contained  in  a  draft 
paper  by  Terrell  and  Aagard.2 


1  Now  Naval  Training  Systems  Center,  Code  10. 

2W.  R.  Terrell  &  3.  Aagard,  J.  (1983,  December).  A  computer  data  base  and 
procedure  for  determining  the  amenability  of  Navy  technical  training  courses  to  computer 

based  instruction  (Draft  technical  report).  Orlando,  FL:  Training  Analysis  and  Evaluation 

Group  (TAEG). 
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Data  Analysis 


The  survey  responses  were  summarized  in  terms  of  frequencies,  percentages,  and 
averages.  The  data  were  sorted  by  type  of  school  (A -school  or  advanced  C-&-F -schools), 
occupational  group  (electrical,  mechanical,  clerical,  operator,  and  team),  and  course 
duration.  Not  all  statistics  derived  from  these  sorts  are  reported  due  to  the  large  volume 
and/or  small  sample  sizes  resulting  from  some  breakdowns.  Table  1  presents  a  breakdown 
of  the  number  of  responses  according  to  the  sorting  dimensions.  Appendix  B  lists  the 
individual  course  titles  by  occupational  group. 


Table  1 

Sample  Frequencies  by  Type  of  School,  Occupational  Group,  and  Course  Duration 


Category 

Sample  Frequencies 

Occupational  Group 

Electrical 

Mechanical  Clerical 

Operator 

Team 

Other 

Total 

Type  of  School 

A 

17 

20 

ii 

14 

0 

2 

64 

C-&-F 

_3 

_9 

29 

12 

17 

J_ 

71 

Total 

20 

29 

40 

26 

17 

3 

135 

Course  Duration 

1-5  days 

0 

1 

15 

7 

17 

1 

41 

6-20  days 

4 

8 

13 

4 

0 

2 

29 

21-40  days 

2 

8 

5 

3 

0 

0 

20 

41-80  days 

6 

11 

7 

5 

0 

0 

29 

81+  days 

_8 

_1 

_0 

_7 

_0 

_0 

16 

Total 

20 

29 

40 

26 

17 

3 

135 

Course  Duration  (days) 

1  to  5 

6  to  20 

21  to  40 

41  to  80 

81+ 

Total 

Type  of  School 

A 

1 

7 

15 

27 

14 

64 

C-&-F 

40 

22 

_5 

_2 

_2 

71 

Total 

41 

29 

20 

29 

16 

135 

Note.  See  Appendix  B  for  a  complete  list  of  survey  courses. 
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SURVEY  FINDINGS 


Course  and  Student  Characteristics 


Tables  2,  3,  and  4  present  course  and  student  characteristics  by  type  of  school, 
occupational  group,  and  course  duration  respectively.  A-school  course  managers  reported 
about  twice  as  many  modules  per  course  as  did  C-&-F -school  managers,  reflecting  the 
generally  longer  duration  of  A-schools.  These  longer  duration  courses  were  in  the 
operator,  electrical,  and  mechanical  occupational  groups.  The  number  of  modules  a 
course  contains  indicates  the  scope  of  effort  a  major  course  revision  would  entail.  Since  a 
few  courses  divided  modules  into  many  small  segments,  both  the  median  (middle  score 
when  ranked)  and  the  arithmetic  mean  for  number  of  modules  are  reported. 

Tables  2,  3,  and  4  also  present  the  average  number  of  students  and  instructors  for 
both  classrooms  and  laboratories.  Classroom  student/instructor  ratios  differed  very  little 
across  the  sample,  except  for  the  large  ratio  for  team  (training)  schools  (Table  3).  The 
ratios  can  reflect  the  cost  effectiveness  for  individual  courses  and,  where  appropriate, 
may  also  indicate  a  need  to  use  CBI  to  compensate  for  instructor  shortfalls. 

A  relatively  small  percentage  of  courses  in  the  total  sample  (12.6%)  currently  use 
CBI.  A-schools  were  four  times  as  likely  as  C-<Jc-F-schools  to  use  CBI.  Among  the 
occupational  groups,  the  most  frequent  use  of  CBI  was  in  the  electrical  group.  Over  half 
of  the  CBI  use  was  in  schools  of  greater  than  40  days  in  duration.  Both  the  mean  and 
median  number  of  modules  using  CBI  are  reported.  Again  the  number  of  modules  depends 
upon  how  the  courses  divided  their  material.  The  17  courses  currently  using  CBI  (listed  in 
Appendix  C)  used  the  following  computer  equipment3  at  the  time  of  the  survey:  Apple, 
TRS-80,  and  DEC  Rainbow. 

Courses  presently  using  computers  in  individual  elements  of  the  instruction  or  course 
management  might  be  strong  candidates  for  CBI  consideration.  However,  the  incompati¬ 
bility  of  courseware  or  management  software  programmed  in  different  computer 
languages  for  differing  computers  would  require  detailed  case-by-case  analysis.  At  the 
time  these  survey  data  were  collected,  contracts  for  "Navy  standard"  microcomputers 
were  just  coming  into  effect;  therefore,  none  of  the  surveyed  courses  used  any  of  these 
machines.  Since  that  time,  four  Navy  standard  microcomputers  have  become  available— 
none  of  them  totally  compatible  with  each  other. 

Student  Scheduling 

Tables  2,  3,  and  4  present  the  average  number  of  working  days  students  waited  before 
the  start  of  a  course;  that  is,  holding  time.  The  percentage  of  course  managers  reporting 
some  holding  time  was  30  percent  overall  (N=40)  and  55  percent  for  A-schools,  and  was 
largest  for  long  duration  schools.  Overall,  course  managers  reported  about  a  week's 
holding  time  for  courses  that  had  other  than  zero  or  "no  wait."  These  tended  to  be  longer 
duration  A-school  courses  in  the  electrical,  mechanical,  and  operator  occupational  groups. 
Courses  with  long  average  holding  times  for  students  might  use  some  form  of  CBI  for 
course-relevant  pretraining,  remediation,  enrichment,  or  other  form  of  individualization. 


3  Identification  of  the  equipment  is  for  documentation 
endorsement. 


only  and  does  not  imply  any 


Table  2 


Course  and  Student  Characteristics  by  Type  of  School 


Course  Managers'  Responses 

Questionnaire  Item 

A-School 

N  =  64 

C-&-F  -School 

N  =  71 

Total 

N  =  135 

Number  of  modules  in  course 

Mean 

Median 

23.4  (62) 

12.5  (62) 

11.3  (48) 
6.5  (48) 

19.2  (110) 
9.5  (110) 

Number  in  average  classroom 

Students 

Instructors 

Student/instructor  ratio 

26.8  (62) 
1.3  (62) 
22.1  (62) 

28.4  (69) 
1.5  (69) 
23.2  (69) 

27.6  (131) 
1.37  (131) 

22.7  (131) 

Number  in  average  laboratory 

Students 

Instructors 

Student/instructor  ratio 

21.5  (56) 
3.4  (56) 
9.7  (56) 

29.7  (59) 
2.9  (59) 
11.4  (59) 

25.7  (115) 
3.15  (115) 
10.6  (115) 

Percent  of  courses  using  CBI 

20.3%  (13) 

5.6%  (4) 

12.6%  (17) 

Number  of  modules  using  CBI 

Mean 

Median 

28.7  (13) 
9.0  (13) 

8.0  (4) 

3.5  (4) 

23.8  (17) 
7.0  (17) 

Student  "holding  time"  prior  to  course 

Percent  of  courses 

Average  number  of  working  days 

55%  (35) 
6.2  (35) 

7%  (5) 

4.8  (5) 

30%  (40) 
6.03  (40) 

Percent  of  courses  with  severe  student  "wait 
time"  for  access  to  laboratory  equipment 

20.3%  (13) 

7.0%  (5) 

13.3%  (18) 

Average  percent  students  reaching  module 
test  criterion  on  first  attempt 

83.3%  (52) 

90.1%  (40) 

86.2%  (92) 

Average  days  completion  time  for 

FAST  students 

SLOW  students 

58.9  (57) 

70.9  (57) 

17.8  (42) 
20.1  (42) 

41.43  (99) 
49.36  (99) 

Difference 

12.04 

2.36 

7.93 

Notes. 


1.  N  =  total  number  of  course  managers  per  type  of  school. 

2.  Averages  are  arithmetic  means,  except  for  skewed  distributions  where  medians 
(midpoints)  are  also  provided. 

3.  The  actual  number  of  course  managers  who  responsed  is  provided  in  parentheses 
after  the  response. 
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Table  3 


Course  and  Student  Characteristics  by  Occupational  Group 


Course  Managers'  Respo 

nses 

Electrical 

Mechanical 

Clerical 

Operator 

Team 

Questionnaire  Item 

N  = 

20 

N  = 

29 

N  =  40 

N  =  26 

N  =  27 

Number  of  modules  in  course 

Mean 

20.4 

(19) 

16.9 

(22) 

13.4  (34) 

35.7  (23) 

4.9  (10) 

Median 

12.0 

(19) 

10.5 

(22) 

7.0  (34) 

12.0  (23) 

4.5  (10) 

Number  in  average  classroom 

Students 

25.5 

(19) 

22.7 

(29) 

28.9  (40) 

24.1  (26) 

42.0  (16) 

Instructors 

1.4 

(19) 

1.2 

(29) 

1.5  (40) 

1.2  (26) 

1.6  (16) 

Student/instructor  ratio 

18.9 

(19) 

21.1 

(29) 

21.5  (40) 

21.9  (26) 

34.6  (16) 

Number  in  average  laboratory 

Students 

18.9 

(17) 

20.6 

(27) 

22.9  (29) 

23.1  (24) 

47.3  (17) 

Instructors 

2.8 

(17) 

2.7 

(27) 

2.2  (29) 

4.8  (24) 

3.1  (17) 

Student/instructor  ratio 

8.7 

(17) 

9.0 

(27) 

13.3  (29) 

8.8  (24) 

13.1  (17) 

Percent  of  courses  using  CBI 

30% 

(6) 

3.4%  (1) 

7.5%  (3) 

15.4%  (4) 

11.8%  (2) 

Number  of  modules  using  CBI 

Mean 

9.3 

(6) 

16.0 

(1) 

5.0  (3) 

74.8  (4) 

5.0  (2) 

Median 

8.0 

(6) 

16.0 

(1) 

2.0  (3) 

16.0  (4) 

2.5  (2) 

Student  "holding  time"  prior 

to  course 

Percent  of  courses 

40% 

(8) 

62% 

(18) 

12.5%  (5) 

31%  (8) 

0%  (0) 

Average  number  of  working  days 

10.1 

(8) 

4.7 

(18) 

4.6  (5) 

6.3  (8) 

-  (0) 

Percent  of  courses  with  severe 
student  "wait  time"  for  access 

to  laboratory  equipment 

25.0%  (5) 

10.3%  (3) 

12.5%  (5) 

15.4%  (4) 

0%  (0) 

Average  percent  students  reaching 
module  test  criterion  on  first 

attempt 

79.2%  (14) 

90.9%  (27) 

88.2%  (23) 

79.9%  (21) 

97.4%  (5) 

Average  days  completion  time  for 

FAST  students 

87.2 

(17) 

37.4 

(27) 

22.0  (27) 

51.2  (20) 

3.2  (6) 

SLOW  students 

95.8 

(17) 

42.9 

(27) 

25.6  (27) 

66.9  (20) 

3.2  (6) 

Difference 

13.1 

5.4 

3.6 

15.7 

0.0 

Notes. 


1.  N  =  total  number  of  course  managers  per  occupational  group. 

2.  Averages  are  arithmetic  means,  except  for  skewed  distributions  where  medians  (midpoints)  are 
also  provided. 

3.  The  actual  number  of  course  managers  who  responsed  is  provided  in  parentheses  after  the 
response. 
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Table  4 


Course  and  Student  Characteristics  by  Course  Duration 


Course  Managers 

Resp< 

onses 

1-5  days 

6-20  days 

21-40  days 

41-80  days 

81+  days 

Questionnaire  Item 

N  = 

41 

N  = 

29 

N  = 

20 

N  = 

29 

N  = 

16 

Number  of  modules  in  course 

Mean 

5.5 

(28) 

16.8 

(20) 

18.9 

(19) 

18.9 

(28) 

49.1 

(15) 

Median 

5.0 

(28) 

10.0 

(20) 

11.0 

(19) 

13.0 

(28) 

13.0 

(15) 

Number  in  average  classroom 

Students 

33.2 

(39) 

28.6 

(29) 

25.2 

(19) 

24.1 

(28) 

21.6 

(16) 

Instructors 

1.6 

(39) 

1.4 

(29) 

1.4 

(19) 

1.2 

(28) 

1.1 

(16) 

Student/instructor  ratio 

26.2 

(39) 

22.4 

(29) 

19.7 

(19) 

21.8 

(28) 

20.8 

(16) 

Number  in  average  laboratory 

Students 

36.0 

(34) 

23.9 

(22) 

21.8 

(16) 

20.4 

(27) 

19.0 

(16) 

Instructors 

2.7 

(34) 

3.6 

(22) 

3.1 

(16) 

2.4 

(27) 

4.9 

(16) 

Student/instructor  ratio 

12.6 

(34) 

8.9 

(22) 

11.7 

(16) 

10.8 

(27) 

7.1 

(16) 

Percent  of  courses  using  CBI 

7.3%  (3) 

10.3%  (3) 

5.0%  (1) 

13.8%  (4) 

37.5%  (6) 

Number  of  modules  using  CBI 

Mean 

4.0 

(3) 

13.0 

(3) 

9.0 

(1) 

12.75  (4) 

49.0 

(6) 

Median 

5.0 

(3) 

16.0 

(3) 

9.0 

(1) 

6.50  (4) 

8.0 

(6) 

Student  "holding  time"  prior 
to  course 

Percent  of  courses 

2.4%  (1) 

10% 

(3) 

50% 

(10) 

45% 

(13) 

81% 

(13) 

Average  number  of  working  days 

5.0 

(1) 

8.6 

(3) 

3.9 

(10) 

5.7 

(13) 

7.5 

(13) 

Percent  of  courses  with  severe 
student  "wait  time"  for  access 

to  laboratory  equipment 

7.3%  (3) 

3.5%  (1) 

25% 

(5) 

20.7%  (6) 

18.8%  (3) 

Average  percent  students  reaching 
module  test  criterion  on  first 
attempt 

Average  days  completion  time  for 

95.1 

(21) 

91.7 

(20) 

88.8 

(16) 

81.6 

(23) 

67.1 

(12) 

FAST  students 

3.5 

(23) 

31.7 

(20) 

25.4 

(17) 

58.0 

(25) 

107.4 

(14) 

SLOW  students 

3.5 

(23) 

35.6 

(20) 

28.5 

(17) 

71.3 

(25) 

130.6 

(14) 

Difference 

0 

3.9 

3.1 

13.3 

23.1 

Notes. 


1.  N  =  total  number  of  course  managers  per  course  duration  category. 

2.  Averages  are  arithmetic  means,  except  for  skewed  distributions  where  medians  (midpoints)  are 
also  provided. 

3.  The  actual  number  of  course  managers  who  responsed  is  provided  in  parentheses  after  the 
response. 
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About  13  percent  of  the  course  managers  reported  a  severe  student  "wait  time"  for 
access  to  laboratory  equipment.  A-school  course  managers  reported  a  much  higher 
severity  of  this  problem  (20.3%)  than  did  C-&-F-course  managers  (7%)  with  the  greatest 
percentage  reported  for  electrical  schools  and  long  duration  courses  (Tables  2,  3,  and  4). 
Severe  wait  times  for  equipment  in  some  laboratories  might  be  remedied  with  more 
efficient  computerized  scheduling  of  student  activities.  In  others,  less  expensive  CBI 
systems  might  be  employed  as  "ancillary"  or  auxiliary  trainers  to  teach  related  informa¬ 
tion  or  component  skills  pertinent  to  the  more  expensive  trainers  for  which  access  is 
limited.  For  example,  a  videotape  introduction  might  give  waiting  students  an  overview 
or  summary  of  procedural  steps  or  they  could  receive  componential  practice  on  the 
effects  of  testing  points  in  a  circuit. 

Student  Progress 

Courses  in  which  a  substantial  number  of  students  are  set  back  or  do  not  reach 
criterion  on  the  first  attempt  complicate  student  management.  Students  in  these  courses 
may  require  additional  instructional  delivery  and  more  individual  attention.  Therefore, 
these  courses  are  good  candidates  for  both  CM  I  and  CAI,  particularly  for  individualized 
remediation  programs.  In  the  total  course  sample,  86  percent  of  the  students  reached 
criterion  on  first  attempt;  the  lowest  percentage  was  79  percent  for  the  electrical  and 
operator  schools;  and  the  highest,  97  percent  for  team  schools.  A  somewhat  lower 
percentage  of  students  reached  criterion  on  module  tests  on  the  first  attempt  in  A-schools 
(83%)  than  in  C-&-F -schools  (90%).  The  lower  success  of  students  in  the  electrical  and 
operator  courses  was  associated  with  longer  duration  courses. 

Courses  in  which  there  is  a  disparity  between  completion  times  for  "fast"  and  "slow" 
students  are  also  good  candidates  for  CBI  such  as  management,  or  remedial  or  enrichment 
programs.  Overall,  the  difference  in  completion  times  between  fast  and  slow  students 
was  about  8  days.  This  difference  was  largest  (13-15  days)  in  electrical  and  operator  A- 
school  courses  and  increased  with  course  duration  (Tables  2,  3,  and  4). 

Instructional  Methods  Employed 

The  instructional  media  profile  of  a  course  indicates  how  complex  the  course  is  to 
manage  and  to  convert  to  CBI.  Table  5  presents  the  instructional  methods  employed  by 
our  course  sample  in  terms  of  the  percentage  of  the  135  courses  that  use  each 
instructional  method  and  the  estimated  percentage  of  time  that  the  courses  using  each 
instructional  method  actually  devote  to  this  method.  The  surveyed  courses  fall  into  three 
groups:  76  percent  usage  (lecture  and  lab)  for  40  percent  of  the  course  time,  56  to  62 
percent  usage  (test,  films  or  videotapes,  and  demonstration)  for  5  to  10  percent  of  course 
time,  22  to  35  percent  (self-study,  discussion,  tutoring,  and  drill  and  practice)  for  7  to  30 
percent  of  course  time. 

Figure  1  presents  these  data  by  type  of  school  and  occupational  group.  The  A-schools 
use  laboratories,  tests,  and  tutoring  more;  while  the  C-<5c-F-schools  use  lectures  more. 
When  broken  down  by  occupational  group,  the  team  courses  are  notable  in  not  using  tests, 
self-study,  or  tutoring.  For  those  courses  actually  using  an  instructional  method,  the 
percent  of  time  devoted  to  self-study  is  higher  for  A-schools,  while  the  C-&-F -schools 
devote  more  course  time  to  discussion  and  drill  and  practice.  With  the  exception  of 
demonstrations  in  team  courses,  all  of  the  courses  spend  less  than  10  percent  of  course 
time  on  tests,  films  or  videotapes,  demonstrations,  and  tutoring.  Electrical  and  clerical 
groups  appear  to  devote  more  time  to  self-study  and  discussion  more  than  other  groups. 
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Courses  Using  Instructional  Method  (%) 


Table  5 


Instructional  Methods  Used  by  Surveyed  Courses 


Instructional  Method 

Surveyed 

Courses 

Using 

Method  (%) 

Estimated 
Time  Using 
Method  (%) 

Lecture 

76 

40 

Laboratory 

76 

40 

Tests 

62 

9 

Films  or  videotapes 

58 

5 

Demonstrations 

56 

10 

Self-study  (print  materials)  35 

30 

Discussions 

33 

27 

Tutoring 

30 

7 

Other  (drill  <$c  practice) 

22 

26 

Note.  N  =  135. 
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Figure  1.  Use  of  instructional  methods  by  school  type  and  occupational  group. 
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Many  of  the  instructional  methods  in  Table  5  and  Figure  1  can  be  converted  to  CBI. 
Procedures  or  steps  that  are  used  frequently  are  often  simulated  in  laboratories. 
Examples  of  applications  amenable  to  CAI  techniques  are  videodisc  presentations  of 
equipment  to  be  operated  or  diagnosed  during  malfunction;  administration  and  scoring  of 
tests,  which  are  common  to  many  of  the  courses;  and,  when  student  throughput  is  large 
enough,  combining  material  currently  in  films  or  videotapes  with  computer  graphics  in  an 
interactive  videodisc  presentation  that  allows  branching  based  upon  student  responses. 
Drill  and  practice  can  be  easily  automated,  because  it  involves  repetitive  presentation  of 
well  defined  to-be-remembered  items  or  knowledge  tests.  (The  response  percentages 
shown  for  drill  and  practice  may  not  be  entirely  accurate  because  respondents  identified 
it  in  the  "other"  catogory.) 

Learning  Objectives 

Table  6  ranks  the  instructional  approach  called  for  by  the  learning  objectives  of  the 
courses  surveyed  in  terms  of  the  percentage  of  courses  requiring  that  approach.  The 
instructional  approaches  form  two  groups  of  "high"  (49-63%)  and  "low"  (18-20%)  require¬ 
ments.  Figure  2  shows  these  responses  broken  down  by  school  type  and  occupational 
group. 


1.  Simulation.  The  high  percentage  of  courses  requiring  simulation  reflects  the 
large  number  of  laboratories  with  high  fidelity  training  devices,  interactive  scenarios  with 
a  complex  catalog  of  consequences,  manipulative  models,  etc.  Accordingly,  a  high 
percentage  of  mechanical,  operator,  and  team  courses  require  simulation.  As  noted 
previously,  this  finding  suggests  an  application  for  CBI. 

2.  Human  interaction.  Modules  within  a  course  requiring  human  interaction  (i.e., 
teams,  discussion  groups,  scenario  role  playing,  etc.)  were  most  frequent  for  team 
occupations,  with  other  groups  reporting  lower  percentages.  Generally,  such  human 
interaction  is  not  considered  suited  for  CAI,  but  it  would  be  applicable  for  computer- 
based  tracking  of  student  progress  and  maintaining  course  records.  CAI  could  be  used  for 
video  re-enactments  of  team  situations  with  stop  points  to  question  an  individual  learner 
about  courses  of  actions  at  important  team  interaction/sequence  points.  Team  situations 
involving  electrical  or  audio  communication  (not  face-to-face)  might  also  be  simulated  in 
CAI. 


3.  Extensive  drill  and  practice.  Learning  objectives  requiring  drill  and  practice 
were  most  frequently  cited  by  operator  and  team  course  managers.  Many  types  of  drill 
and  practice  material  are  particularly  good  candidates  for  application  of  CBI  as,  for 
example,  when  drill  and  practice  is  so  extensive  or  responses  so  precise  that  the  student- 
instructor  ratio  or  human  patience  make  the  traditional  classroom  setting  impractical. 
Drill  and  practice  can  include  memorizing  extensive  or  complex  knowledge  bases  or 
repeating  or  performing  procedures  with  equipment,  team  situations,  and  other  job 
sequences  that  require  the  learners  to  repeat  procedures  until  they  become  well 
practiced. 

4.  Variable  responses.  The  term  "variable  response"  refers  to  complex  responses 
that  vary  according  to  the  context  of  the  interaction.  A  situation  may  involve  complex 
"if-then"  relations  that  do  not  always  have  a  required  order  or  sequence  for  performing 
the  task.  The  resulting  large  number  of  possible  action  sequences  may  make  cataloging  a 
set  of  fixed  responses  impractical.  For  example,  many  equipment  problem  solving 
situations  require  variable  solution  paths.  Another  example  would  be  with  many  combat 
information  center  (CIC)  or  antisubmarine-warfare  (ASW)  team  situations,  where,  for 
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Table  6 


Learning  Objective  Requirements 


Do  the  Learning  Objectives  Require: 

Yes 

Responses  (%) 

No 

NAa 

Simulation 

63 

35 

2 

Human  interaction 

61 

39 

0 

Extensive  drill  and  practice 

54 

43 

3 

Variable  responses 

49 

51 

0 

Automated  cues  and  prompts 

20 

76 

4 

Computer  utilization 

18 

81 

1 

Note.  N  =  135. 


aNA  =  not  applicable. 
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Figure  2.  Learning  objective  requirements  by  school  type  and  occupational  group. 
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example,  one  person  errs  and  another  makes  a  compensatory  move  leading  to  a  correct 
solution.  Modules  within  a  course  requiring  variable  responses  are  not  easily  programmed 
for  CAI,  but  computer-based  management  may  be  used  for  tracking  and  keeping  course 
records. 


•5.  Automated  cues  and  prompts.  Course  modules  requiring  automated  cues  and 
prompts  were  most  frequently  cited  by  operator  courses  and  infrequently  by  others.  The 
selection  of  CAI  simulation  should  be  considered  for  course  modules  involving  interaction 
with  automated  equipment  in  which  the  operator  has  little  or  no  control  over  the 
occurrence  of  events  or  their  input  rate.  Often  the  operator  is  also  subject  to  additional 
requirements  by  other  persons  as  a  consequence  of  the  nature  or  rate  of  the  operator's 
responses.  For  example,  the  rate  at  which  a  Naval  Tactical  Data  System  tracking 
operator  updates  a  track  depends  on  the  system  latency  and  the  number  of  commands  in 
the  queue.  The  number  of  commands  will  be  affected  by  the  input  from  other  consoles  in 
the  system  and  voice  channel  requests  for  information. 

6.  Computer  utilization.  Learning  objectives  requiring  computer  utilization  were 
most  frequently  cited  by  managers  of  electrical  and  operator  A-school  courses.  Modules 
requiring  a  computer  for  computations,  information  retrieval,  or  other  similar  problem¬ 
solving  techniques  suggest  an  amenability  with  CAI.  Other  uses  are  related  to  the  same 
issues  discussed  for  automated  cues  and  prompts. 

Test  Methods 


Responses  about  the  test  methods  employed  by  the  courses  are  separated  in  Table  7 
according  to  whether  the  existing  test  methods  are  computerized,  manual,  or  not 
applicable.  These  in  turn  are  subdivided  according  to  whether  the  course  managers  were 
satisfied  (OK)  or  were  not  satisfied  (NO)  with  the  test  methods.  Figure  3  presents  the  use 
of  test  methods  by  school  type  and  occupational  group  as  well  as  whether  the  course 
managers  considered  the  test  method  inadequate  and  adequate.  (Some  of  the  sample  sizes 
are  quite  small.) 

In  general,  manual  test  methods  are  used  far  more  frequently  than  computer 
methods,  and  course  managers'  satisfaction  outweighs  dissatisfaction.1*  Where  computers 
are  used,  satisfaction  seems  to  outweigh  dissatisfaction,  although  this  is  also  true  of  the 
manual  methods. 

The  greatest  current  use  of  computers  is  for  general  administrative  test  scoring, 
analysis,  and  recording.  Figure  3  shows  the  highest  usage  of  computer  testing  is  by  A- 
schools  and  the  electrical  and  mechanical  occupational  groups.  The  electrical  group  also 
used  computerized  test  methods  more  often  than  other  test  methods.  Courses  requiring  a 
significant  number  of  instructional  man-hours  for  test  scoring,  test  analysis,  and  test 
recording  usually  are  amenable  to  CBI.  Courses  that  require  extensive  testing  or 
maintain  substantial  test  item  banks  are  excellent  applications  of  CMI.  Constant  test 
availability  and  the  ability  to  systematically  or  randomly  generate  alternate  test  forms 
are  some  of  the  benefits  associated  with  automating  test  processes. 


**This  finding  is  generally  confirmed  when  the  course  managers'  satisfaction  and 
dissatisfaction  with  each  test  method  are  calculated  separately  for  computer  and  manual 
test  methods.  See  Table  D-l. 
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Table  7 

Test  Methods  Employed 


Test  Methods  Used 

Responses  (%) 

Computer 

OK  FfO 

OK 

Manual 

NO 

Wanta 
to  Add 

NAb 

Performance  evaluation 

9 

3 

60 

13 

0 

16 

Pretesting 

9 

2 

16 

14 

6 

53 

Unit/module  tests 

11 

1 

40 

16 

1 

31 

Summary/final  tests 

7 

1 

33 

15 

1 

43 

Remediation  prescription 

7 

1 

27 

13 

1 

50 

Test  item  bank 

5 

1 

34 

16 

4 

39 

Test  scoring 

16 

5 

28 

18 

1 

32 

Test  analysis 

15 

8 

20 

16 

1 

39 

Recording 

16 

4 

32 

23 

1 

24 

Notes. 

1.  Percentages  do  not  always  total  100  due  to  rounding. 

2.  See  Table  D-l  for  alternate  calculations  of  satisfaction  and  dissatisfaction  with 
each  test  method. 

aCourse  manager  would  like  to  add  a  test  method  not  currently  employed. 
bNA  =  not  applicable. 
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Figure  3.  Test  method  use  by  school  type  and  occupational  group. 
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Many  of  the  courses  that  prescribe  remedial  assignments  for  each  student  according 
to  a  diagnosis  of  test  results  should  also  be  amenable  to  CMI.  The  largest  percentages  of 
not-applicable  responses  were  for  pretesting  and  remediation  prescription  (about  50%),  as 
well  as  for  final  tests,  item  banking,  and  test  analysis  (about  40%);  all  indicate  methods 
simply  not  employed  at  present. 


Table  7  shows  that  60  percent  of  the  course  managers  were  satisfied  with  the  existing 
manual  (noncomputerized)  test  method  of  "performance  evaluation."  A  heavy  emphasis  on 
evaluating  real  performance  in  laboratories  suggests  that  computerized  performance 
testing  must  offer  an  improvement  to  top  this  satisfaction  with  existing  techniques. 
Performance  evaluations,  which  would  benefit  from  electrical  scoring  and/or  recording 
with  performance  reports  shown  alphanumerically  and/or  graphically,  suggest  CMI  should 
be  considered.  & 

Special  Course  Problems 

Special  problem  areas  identified  by  the  course  managers  are  shown  in  Table  8  and 
Figure  4.  The  "yes"  responses  in  the  table  fall  in  the  relatively  narrow  range  of  33  to  46 
percent. 

Overall,  39  percent  of  the  course  managers  reported  inadequate  learning  objectives. 
This  was  more  characteristic  of  C-<5c-F-schools  than  A-schools.  Inadequate  terminal 
and/or  enabling  objectives  indicate  that  a  substantial  course  revision  may  be  needed 
before  investing  in  CBI;  however,  this  revision  would  be  a  good  opportunity  for  using  CBI. 
Inadequate  objectives  would  include  objectives  that  do  not  specify  desired  behavior, 
present  insufficient  content  scope  and  depth,  or  have  an  imbalance  in  the  types  of 
objectives  (e.g.,  too  few  recall-fact  objectives  in  relation  to  the  number  of  use-unaided 
procedure  objectives;  cf.,  Montague,  Ellis,  &  Wulfeck,  1983). 

Thirty-nine  percent  of  the  course  managers  reported  curriculum  stability  as  a  special 
problem  area.  Curriculum  instability  refers  to  dynamic  subject  matter  where  lesson 
content  changes  constantly  (e.g.,  because  of  new  equipment).  Updating  subject  matter  for 
CBI  would  require  special  consideration  when  a  course  adds  brand-new  subject  matter. 

A  large  number  of  students  entering  a  course  without  prerequisite  skills  suggest  the 
need  to  develop  remedial  programs  to  correct  the  deficiencies.  CBI  should  be  considered 
as  a  vehicle  for  diagnosing  the  need  and  providing  the  remedial  programs.  Likewise,  the 
specific  deficiencies  in  reading  and  math  (slow  progress,  repeated  test  failures,  and 
excessive  requirements  for  tutoring)  suggest  the  need  for  an  alternate  instructional 
program  or  remediation.  CBI  should  be  considered  as  a  vehicle  for  meeting  the  special 
needs  of  students  with  low  reading  and  math  level  scores.  Figure  4  shows  that  these 
problems  are  generally  more  prevalent  in  A-schools  and  in  electrical,  mechanical,  and,  to 
a  lesser  extent,  operator  occupational  groups.  While  overall  1/3  to  1/2  of  the  course 
managers  reported  these  special  problems,  only  3  percent  (four  courses)  assign  special 
materials  based  on  aptitude  scores  (discussed  later).  This  finding  suggests  that  individual¬ 
ized  remediation  is  not  currently  very  prevalent,  but  has  the  potential  for  substantial 
improvement.  CBI  offers  the  potential  for  individualization,  self-pacing,  and  program- 
controlled  branching  to  special  content  or  repetition  based  on  a  student's  response. 
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Table  8 


Special  Course  Problem  Areas 


Special  Problem  Areas 

Yes 

Responses  (%) 

No 

NAa 

Adequate  learning  objectives 

39 

58 

3 

Curriculum  stability 

39 

60 

1 

Student  entering  skills 

33 

63 

4 

Student  math  ability 

35 

59 

7 

Student  reading  ability 

46 

53 

1 

Notes. 


1.  N  =  135. 

2.  Percentages  do  not  always  total  100  due  to  rounding. 
NA  =  not  applicable. 
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Figure  4.  Special  course  problem  areas  by  school  type  and  occupational  group. 
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Achievement  Distribution 


Table  9  shows  the  distribution  of  student  achievement  as  perceived  by  the  course 
managers.  For  this  question,  the  respondents  were  asked  to  characterize  their  students' 
achievement  by  choosing  one  of  three  student  ability  ratings  (below-average,  average, 
above-average)  or  not  applicable.  In  the  total  sample,  about  half  of  the  course  managers 
judged  the  students'  achievement  to  be  average  with  more  citing  "above"  than  "below" 
average.  The  A-schools  cited  more  below-average  student  achievement  than  the  C-&-F- 
schools,  where  more  above-average  achievement  was  reported.  By  occupational  group, 
about  half  of  the  team  course  managers  (9)  reported  that  rating  student  achievement  was 
not  applicable.  The  largest  number  of  below-average  ratings  was  in  the  operator  group 
and  the  largest  number  of  average  ratings  was  in  the  electrical  and  mechanical  groups. 
Individual  courses  with  average  or  below  average  achievement  distributions,  high  attrition 
and  setback  rates,  and  low  reading  and  math  scores  might  be  candidates  for  remedial 
instruction.  For  above-average  students,  enrichment  might  be  appropriate.  Many  basic 
skills  remediation  and  enrichment  situations  would  be  amenable  to  CBI  because  this  type 
of  instruction  is  widely  available  in  the  general  world  of  education. 


Table  9 

Perceived  Distribution  of  Student  Achievement  by 
School  Type  and  Occupational  Group 


Student  Achievement  (%) 

Category 

a 

n 

Below  Above 

Average  Average  Average 

NAb 

Occupational  Group 

Electrical 

(20) 

10  (2) 

65  (13) 

25  (5) 

0  (0) 

Mechanical 

(29) 

7  (2) 

62  (18) 

31  (9) 

0  (0) 

Clerical 

m 

8  (3) 

45  (18) 

35  (14) 

13  (5) 

Operator 

(26) 

19  (5) 

42  (11) 

31  (8) 

8  (2) 

Team 

(17) 

0  (0) 

12  (2) 

35  (6) 

53  (9) 

School  Type 


A-schools 

C-<5c-F-schools 

(64) 

(71) 

16  (10) 

6  (4) 

56  (36) 

37  (26) 

27  (17) 

35  (25) 

2 

23 

(1) 

(16) 

Total  Sample 

(135) 

10  (14) 

46  (62) 

31  (42) 

13 

(17) 

Notes. 


1.  The  actual  number  of  responses  is  given  in  parentheses  after  the  response 
percentages. 

2.  Not  all  percentages  total  100  due  to  rounding, 
n  =  Total  number  of  course  managers  per  category. 

NA  =  not  applicable. 
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Use  of  Aptitude  Scores 


Only  four  A-school  course  managers  (3%)  assigned  their  students  special  materials 
based  on  aptitude  scores:  (1)  Operations  Specialist  A  (A221-0011)  using  Gates-MacGinitie 
Reading  Test,  (2)  Academic  Remedial  Training  (A950-0061)  using  Gates-MacGinitie 
Reading  Test,  (3)  Job-oriented  Basic  Skills,  Operations  Strand  II  (A  100-0059)  using  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB)  and  Gates-MacGinitie  Reading  Test,  and 
(4)  Job-oriented  Basic  Skills,  Electrical  Strand  IV  (A100-0060)  using  ASVAB  and  Gates- 
MacGinitie  Reading  Test.  Courses  assigning  special  materials  to  students  based  on  their 
aptitude  scores  are  good  candidates  for  CBI  if  they  teach  basic  skills  such  as  technical  or 
general  vocabulary,  reading,  and  mathematics. 

Computer  Use 

Twenty-six  percent  (N=35)  of  the  course  managers  said  their  students  would  use  a 
computer  on  the  job  after  completing  this  course;  73  percent  (N=99)  said  they  would  not 
(1%,  N=1  said  not  applicable).  Figure  5  shows  that  three  to  four  times  as  many  A-school 
as  C-<5c-F-school  course  managers  answered  that  their  students  would  use  computers. 
Electrical  and  operator  schools  were  prominent  in  on-the-job  computer  use.  The  pattern 
of  the  responses  to  this  question  is  quite  similar  to  that  shown  for  the  learning  objective 
of  computer  utilization  (see  Figure  2).  However,  the  response  percentages  for  the 
learning  objective  question  in  Figure  2  were  generally  less  than  those  for  the  present  on- 
the-job  use  question  (7%  overall;  15.6%  for  A-schools;  0%  for  C-3c-F-schools).  If  course 
graduates  will  use  a  computer  in  their  work,  consideration  should  be  given  to  training 
them  with  computers.  Training  for  computer  use  on  the  job  can  be  amenable  to  CBI. 
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Figure  5.  Student  use  of  computers  on  the  job  after  completing  course. 
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Of  the  135  course  managers,  55  responded  to  the  question,  "What  modules  in  the 
course  should  be  considered  a  high  priority  for  some  form  of  computer-assisted  instruc¬ 
tion?"  and  27  percent  (N=37)  nominated  at  least  one  module  as  suitable  for  CBI  (Table  10). 
The  specific  type  of  modules  nominated  were  mostly  for  remediation,  testing,  and 
management  functions. 


Table  10 

High  Priority  Modules  for  Computer-assisted  Instruction 


Categorized  Responses 

Number  of 
Responses 

Nominated  at  least  one  module 

37 

Wanted  remediation  in  math  and  reading 

5 

Wanted  test  administration  and  processing 

4 

Indicated  management  functions,  like  record  keeping 
and  student  registration  and  scheduling 

7 

Develop  "computer  literacy" 

2 

Total 

55 

At  the  end  of  each  questionnaire,  respondents  were  asked  the  open-ended  question, 
"If  computer  support  was  available  for  your  school,  what  functions  would  have  first 
priority?"  They  cited  the  following  three  functions  most  frequently: 

1.  Computer  support  for  course  managers  (97%). 

2.  Computer-based  drill  and  practice,  test  scoring  and  recording,  and  student 
tracking  (90%). 

3.  Computer-based  simulation  and/or  position  training5  (68%). 

Again,  these  response  percentages  show  that  course  management  and  recording,  tracking, 
drill  and  practice,  and  simulation  were  all  clearly  of  high  interest  to  the  course  managers. 
Computers  are  viewed  more  as  instructional  and  managerial  support  devices  than  as 
primary  instructional  systems. 

Adequacy  of  Course  Management 

Responses  about  the  use  of  three  management  processes  are  separated  in  Table  11 
into  existing  computer  or  manual  test  methods,  which  are  subdivided  according  to  whether 
or  not  the  course  managers  are  satisfied  (OK  and  NO).  The  course  managers  responded 


5  Position  training  is  usually  associated  with  team  training  and  concerns  the  training 
of  individual  operator  skills  for  a  station  in  the  team  situation. 
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most  frequently  that  they  wanted  to  add  a  management  method  or  that  they  were 
satisfied  with  the  existing  manual  method.  In  general,  the  satisfaction  outweighed  the 
dissatisfaction  for  each  of  the  two  methods.  This  finding  is  generally  confirmed  when  the 
course  managers'  satisfaction  with  their  course  management  process  is  calculated 
separately  for  computerized  and  manual  management  processes.  See  Table  D-2. 


Table  11 

Course  Management  Processes 


Responses  (%) 


Computer 

Manual 

Want 

Management  Processes  Used 

OK 

NO 

OK 

NO 

to  Add 

NAa 

Student  time/test-attempt 
records 

5 

2 

36 

19 

36 

2 

Student  scheduling 

16 

7 

30 

10 

36 

1 

Resource  scheduling 

7 

4 

44 

16 

28 

1 

Notes. 


1.  N  =  135. 

2.  See  Table  D-l  for  alternate  calculations  of  satisfaction  and  dissatisfaction  with 
the  present  course  management  process. 

NA  =  not  applicable. 


Figure  6  shows  the  responses  from  Table  11  by  school  type  and  occupational  group  as 
well  as  whether  the  course  managers  considered  their  management  process  inadequate  and 
adequate.  (Some  of  the  sample  sizes  have  become  small.)  Figure  6  shows  that 
computerized  management  is  not  widely  used  although  A-schools  use  it  more  than  C-&-F- 
schools;  their  use  of  manual  methods  is  about  the  same.  Likewise,  electrical  and 
mechanical  courses  use  more  computers  than  the  other  groups. 

Computers  can  easily  automate  the  management  processes  of  record  keeping  and 
tracking  of  student  progress.  Courses  that  would  benefit  from  a  record  of  student 
instruction  time,  time  in  tests,  test  attempts,  and  automated  scoring  and  tabulation 
should  be  amenable  to  CMI.  Some  of  this  information  could  also  be  used  online  during  CAI 
to  provide  feedback  to  the  learner. 
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Figure  6.  Use  of  course  management  processes  by  school  type  and  occupational  group. 


The  scheduling  of  students,  instructors,  classrooms,  laboratories,  materials,  and 
equipment  and  maintaining  an  adequate  supply  of  expendables  is  very  complex  and  should 
be  amenable  to  CMI.  Courses  required  to  generate  many  reports  should  be  considered  for 
CMI  (e.g.,  equipment  use,  time,  tracking  of  classified  materials,  dispensing  supplies  such 
as  drugs  or  ammunition,  etc.). 

Management  Documents 

Course  managers  were  asked  to  give  the  date  of  the  latest  revision  of  the  various 
management  documents  associated  with  their  course.  The  responses  were  divided  into 
three  groups— less  than  3  years  ago,  3  to  5  years  ago,  and  more  than  5  years  ago.  The 
response  percentages  on  Table  12  do  not  include  the  not-applicable  responses.  The  large 
percentages  of  not-applicable  responses  result  from  the  course  managers'  lack  of 
knowledge  (e.g.,  before  they  were  assigned  to  the  duty  station)  or  the  fact  that  the 
document  is  not  used  in  the  conduct  of  the  course.  According  to  this  measure,  the 
curriculum  outline  is  the  document  most  commonly  known  about  by  the  course  managers. 

Although  the  instructional  systems  development  (ISD)  process  (Chief  of  Naval 
Technical  Training,  1981)  specifies  the  revision  frequencies  required  for  course  docu¬ 
ments,  actual  revisions  of  the  surveyed  courses  vary  considerably  from  the  requirements. 
Courses  which  have  recently  undergone  a  full  ISD  revision  may  not  be  good  candidates  for 
major  changes  such  as  CBI  unless  an  evaluation  reveals  deficiencies  in  effectiveness,  a 
major  problem  such  as  instructor  shortfall  occurs,  or  very  significant  cost  benefits. 
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Table  12 


Course  Management  Documents:  Revision  Status 


Document 

Last  Revision  (%) 

NAa 

Responses 

Under  3 

Years 

3  to  5 
Years 

Over  5 
Years 

% 

n 

Project  plan 

59 

38 

3 

42 

57 

Curriculum  outline 

55 

36 

9 

16 

21 

Instructional  management  plan 

76 

22 

1 

47 

63 

Instructor  guide 

68 

26 

6 

45 

61 

Instructional  materials 

69 

26 

5 

44 

59 

Criterion  referenced  tests 

72 

22 

7 

45 

61 

Notes. 


1.  N  =  135. 

2.  Response  percentages  computed  without  the  not-applicable  (NA)  responses. 

3.  Response  percentages  may  not  always  total  100  due  to  rounding. 

aNA  =  not  applicable. 


DISCUSSION  AND  CONCLUSIONS 

This  assessment  sought  to  identify  patterns  and  problems  in  current  instructional 
practices.  Of  particular  concern  was  identifying  where  Navy  training  could  be  improved 
by  using  low-cost  microcomputers  as  tools  for  management  or  instructional  delivery. 

Course  Management 

The  lowest  common  denominator  to  all  of  the  surveyed  courses  was  their  need  for 
some  general  administrative/clerical  computer  support  in  registering,  scheduling,  track¬ 
ing,  score  recording  and  general  record  keeping.  These  functions  differ  in  how  visible 
they  are  in  the  actual  delivery  of  instruction. 

One  visible  instance  of  CMI  is  guiding  students  through  a  course  of  study  by  directing 
them  to  specific  study  activities,  testing  at  various  points,  and  directing  remediation. 
Another  CMI  situation  is  where  a  large  central  computer  scores  off-line  tests  and  then 
provides  students  with  prescriptive  study  assignments  (e.g.,  in  large  scale  Navy  training 
operations  such  as  those  in  Memphis,  Great  Lakes,  Orlando,  and  San  Diego).  The 
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limitation  of  this  CMI  application  is  the  student  throughput  and  the  resultant  economic 
cost/benefit  tradeoff.6 

This  survey  found  that  62  percent  of  the  courses  used  tests  for  approximately  9 
percent  of  the  time  for  the  general  purpose  of  evaluating  performance.  More  of  these 
test  methods  were  manual  than  were  computerized,  with  computers  used  most  for  test 
scoring,  analysis,  and  recording  (15-16%).  Pretesting  and  remediation  prescription,  which 
were  the  least  used  of  the  test  methods  surveyed,  can  be  automated  when  instruc¬ 
tional  delivery  is  computerized.  Testing  functions  in  general  offer  great  potenital  for 
both  computerized  management  and  instruction.  Specific  benefits  include  having  records 
of  test  attempts,  time  in  tests,  instruction  time,  automated  scoring  and  tabulation,  test 
development,  test  scoring,  diagnostic  analysis,  and  remediation  prescription.  These 
benefits  are  more  attractive  in  situations  with  a  high  volume  of  students  and  stable 
curricula. 

Course  management  also  includes  some  more  general  support  activities  that  often 
involve  tedious  clerical  tasks  such  as  scheduling  of  students,  instructors,  and  other 
resources  such  as  training  equipment.  The  survey  showed  these  functions  were  performed 
manually  for  40  to  60  percent  of  the  courses  with  another  28  to  36  percent  wanting  to  add 
them  because  they  currently  did  not  exist  at  all.  Other  clerical  tasks  are  maintaining 
equipment  and  material  inventories,  generating  reports  and  keeping  miscellaneous  records 
(i.e.,  time  equipment  is  used,  classified  materials,  control  of  consumable  supplies  such  as 
drugs  or  ammunition). 

Currently,  many  instances  of  clerical  support  are  most  easily  provided  locally  by 
using  low  cost  microcomputers  and  widely  available  spreadsheet  and  word  processor 
programs.  There  is  no  question  about  the  usefulness  of  computers  for  course  support,  this 
is  merely  an  extension  of  the  general  office  automation  trend  in  all  parts  of  society. 
However,  the  proliferation  of  nonstandard  hardware  and  software  becomes  an  issue  for 
many  of  these  functions  when  documents  or  data  must  be  shared.  Standard  spread- 
sheet/word  processor  programs  might  make  life  a  little  easier  when  these  situations 
arise.7 

Local  or  Navy-wide  development  will  still  be  required  for  specialized  programs  not 
covered  by  the  commercially  available  software.  Software  developed  by  Navy  Personnel 
Research  and  Development  Center  (NAVPERSRANDCEN)  for  an  existing  UNIX8  operating 
system  provides  some  examples,  such  as  a  test  formatting  package  for  the  Personnel 


6 There  is  dissatisfaction  with  the  current  "Navy  CMI"  system  (i.e.,  the  test  scoring 
system  based  on  a  mainframe  computer  at  Chief,  Naval  Technical  Training,  Naval  Air 
Station,  Memphis).  However,  this  satisfaction,  we  think,  reflects  the  need  to  rework 
curricula,  tests,  and  instructional  delivery  methods  rather  than  the  computer  technology 
employed. 

7  Within  the  Navy,  these  programs  seem  to  be  those  that  are  most  popular  in  general 
civilian  use.  Although  not  related  to  training,  the  Naval  Data  Automation  Command 
(NAVDAC)  and  their  regional  facilities  (Naval  Regional  Data  Automation  Centers 
(NARDACs))  have  been  developing  a  set  of  programs  called  "BASIS."  These  programs 
provide  some  software  standardization  for  uses  such  as  making  BOQ  reservations,  issuing 
car  decals,  and  keeping  on-board  personnel  counts.  NARDAC  (Norfolk,  VA)  also  issues  the 
very  useful  quarterly  periodical  Chips  Ahoy  to  keep  Navy  microcomputer  users  abreast  of 
new  hardware,  software,  and  procurement  information. 

8  UNIX  is  a  trademark  of  Bell  Laboratories. 
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Performance  Profile  (PPP)  tables,  which  the  NAVSEA  OD45519  ISD  model  requires 
(Bautista,  Wetzel,  5c  Wulfeck,  1985).  Other  examples  are  frequently  used  local  forms  such 
as  a  cost  projection  sheet  that  automatically  totals  entries  (Lim,  Wetzel,  5c  Wulfeck, 
1985)  or  milestone  charts  used  in  management  documents  (Wulfeck,  McMichael,  5c  Wetzel, 
1985).  These  software  packages  grew  out  of  a  need  to  automate  time  consuming  and 
technically  demanding  procedures  on  an  existing  computer.  The  authoring  instructional 
materials  (AIM)  project  at  NAVPERSRANDCEN  is  currently  developing  similar  software 
to  automate  ISD  procedures  for  Navy  activities  responsible  for  curriculum  development 
and  guide  the  development  process  by  capturing  instructional  development  expertise. 

A  number  of  the  results  reported  here  simply  document  general  background  statistics. 
For  example,  differences  in  class  sizes  and  student/instructor  ratios  generated  little 
comment  since  they  were  all  high  enough  to  make  effective  individual  treatment  a 
problem.  Increased  instructor  resources  might  be  desired  to  achieve  greater  effectiveness 
but  may  not  be  available  locally.  Some  increased  effectiveness  could  result  from  selected 
CMI  applications  to  reduce  management  and  clerical  tasks;  selected  CAI  applications 
could  also  supplement  direct  instruction. 

Instructional  Delivery 

Currently  about  12.6  percent  of  all  courses  surveyed  used  some  form  of  CBI  (20%  in 
A-schools  and  5.6%  in  C-5c-F -schools).  Most  of  these  were  in  electronics  related  schools 
(30%),  where  computer  use  was  also  generally  greatest  for  other  uses  (e.g.,  test  methods 
and  scheduling).  In  assessing  future  interest,  about  27  percent  of  the  courses  nominated 
at  least  one  module  as  suitable  for  CBI.  After  support  for  course  managers,  the  next 
ranked  priorities  for  computer  support  included  CBI  applications. 

Curriculum  stability  was  cited  as  a  special  problem  area  by  39  percent  of  the  course 
managers  because  revisions  are  often  a  natural  consequence  of  updating  courses  to  reflect 
newly  implemented  technologies.  When  training  is  packaged  (as  in  CBI  or  video  CBI)  and 
is  less  dependent  upon  individual  instructors,  there  is  greater  standardization  and  control 
over  the  quality  of  the  instruction  and  its  delivery.  This  feature  becomes  apparent  when 
the  instruction  is  to  be  delivered  at  many  remote  training  sites. 

Another  problem  related  to  the  maintenance  of  course  materials  was  the  inadequacy 
of  learning  objectives,  which  39  percent  of  the  course  managers  cited.  This  reflects  the 
need  for  a  more  systematic  application  of  ISD  principles.  The  computer  per  se  will  help  in 
this  area.  Over  time,  maintenance  of  course  computer  files  will  enable  revisions  to  be 
made  faster  and  easier.  The  danger  is,  however,  that  the  computer  will  simply  produce 
poor  objectives  and  poor  training  faster  and  easier.  Efforts  like  the  AIM  project 
mentioned  earlier,  which  is  directed  at  quality  improvement  not  just  quantity,  are 
therefore  necessary. 

The  survey  also  identified  a  group  of  problems  related  to  student  ability  and  progress, 
which  are  areas  for  which  selected  computer  applications  might  be  adapted.  We  found  33 
to  46  percent  of  the  course  managers  reporting  that  students  had  problems  with  math  and 
reading  ability  and  entering  skills  and  about  14  percent  of  the  students  did  not  reach 
criterion  on  the  first  attempt.  About  36  percent  of  the  managers  wanted  to  track  test 
attempts  and  another  21  percent  were  dissatisfied  with  their  current  tracking.  Overall, 
the  difference  between  fast  and  slow  students  completing  a  course  was  about  8  days;  53 
percent  of  the  managers  reported  pretesting  was  not  applicable,  and  only  3  percent 
assigned  special  materials  based  on  aptitude  scores.  All  of  these  problems  could  be 
addressed  by  various  forms  of  individual  adaptation  or  remediation.  Overall,  pretesting 
(41%)  and  remediation  prescription  (48%)  were  the  least  frequently  used  testing  methods. 


23 


These  data  indicate  a  major  training  deficiency  that  can  only  be  met  by  some  computer 
support  or  by  an  unlikely  increase  in  instructors  and  facilities.  In  general,  CBI  is 
applicable  in  any  situation  calling  for  attention  to  individuals  such  as  not  being  able  to 
call  a  class  together,  the  need  to  prescribe  individual  remediation,  or  scheduling 
difficulty.  For  example,  the  schedule  of  personnel  aboard  ship  may  limit  the  number  of 
persons  available  for  training  in  a  given  block  of  time  to  only  a  few  or  just  one. 

Costly  training  devices  can  act  as  bottlenecks  when  students  have  to  spend 
substantial  time  waiting  for  access  to  limited  quantities  of  laboratory  equipment  (Van 
Kekerix,  Wulfeck,  5c  Montague,  1982).  Long  waits  for  access  to  such  equipment  were 
cited  by  course  managers  in  20  percent  of  the  A-schools,  25  percent  of  electrical  schools 
and  10  to  15  percent  of  the  remaining  non-team  occupation  groups.  Costly  high  fidelity 
laboratory  training  equipment  could  be  supplemented  with  lower-cost  microcomputer 
ancillary  (auxiliary,  supplemental,  or  subordinate)  training  devices.  Ancillary  trainers 
need  not  replace  the  high  cost  equipment;  rather,  only  selected  functions  of  the 
equipment  need  be  simulated.  For  example,  some  aspects  of  equipment  trouble  shooting 
could  be  simulated  by  a  microcomputer  displaying  oscilloscope  patterns  after  inputing  the 
labeled  test  points  found  on  a  printed  schematic.  Or,  a  videotape  could  introduce  the 
equipment  to  be  used  in  a  laboratory.  Ancillary  trainers  could  allow  students  waiting  for 
the  higher  fidelity  trainer  to  be  occupied  more  productively. 

Course  managers  frequently  cited  learning  objectives  calling  for  simulation  (63%)  and 
drill  and  practice  (54%).  Remembering  facts  and  use  of  procedures  were  found  to  be 
particularly  frequent  learning  objectives  in  a  related  survey  of  Navy  training  courses,9 
which  analyzed  learning  objectives  in  terms  of  the  Instructional  Quality  Inventory 
classification  scheme  (Montague,  Ellis,  5c  Wulfeck,  1983).  Taken  together,  such  learning 
objectives  can  be  addressed  particularly  well  by  CAI  programs  designed  to  provide  drill 
and  practice  (remembering  facts  such  as  technical  vocabulary  or  the  information  in  large 
data  bases)  and  the  use  of  procedural  steps  (such  as  operating  a  piece  of  equipment).  CAI 
can  easily  provide  the  most  accepted  usage  of  computers  in  today’s  teaching  environment 
by  providing  the  drill  and  practice  of  mathematical  and  language  studies,  which  might  be 
needed  for  the  entering  or  basic  skills.  Such  programs  have  been  commonly  available  in 
the  general  world  of  education  for  quite  some  time  and  the  nature  of  this  curricula  is  not 
rapidly  changing. 

Many  of  the  CAI  applications  suggested  here  involve  some  general  formula  for 
supervising  the  learning  process.  A  typical  CAI  sequence  involves  presenting  material  for 
review  or  learning  which  may  be  followed  by  practicing  the  facts  or  procedures  that  are 
being  acquired.  Performance  is  assessed  at  various  points  to  test  for  learning  with 
feedback  often  provided.  This  assessment  of  the  student  performance  is  then  compared  to 
some  criterion  for  the  purpose  of  branching  back  to  the  original  instruction  or  explanatory 
tracts  tailored  to  the  misunderstanding  so  that  the  learning  criterion  can  be  attained  in 
later  attempts.  Computer  interaction  forces  more  active  involvement  than  a  lecture  does 
since  not  paying  attention  may  lengthen  the  student's  study  time.  With  conventional 
methods,  branching  within  a  training  manual  might  have  to  include  a  prohibitively  large 
number  of  explanatory  tracks  to  accommodate  differences  among  individuals.  When 
instruction  is  individualized  and  not  tied  to  a  group,  the  instruction  can  be  adjusted  to  the 
learner's  own  pace  or  skill  level,  which  should  include  incentives  to  minimize  dawdling. 
The  general  formula  presented  here  does  not  refer  to  total  self-pacing  in  a  course,  even 
with  such  incentives.  The  same  techniques  can  also  be  used  for  enrichment,  night  study, 
remediation,  laboratory  management,  etc. 


9  Prepublication  findings  (C.  D.  Wetzel,  D.  L.  Van  Kekerix,  5c  W.  H.  Wulfeck). 
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Additional  enhancements  to  this  basic  adaptive  feedback  formula  may  be  employed  to 
simulate  reality  according  to  the  specific  training  applications.  Videodisc  images  or 
changeable  high  resolution  graphics  can  illustrate  complex  principles  not  easily  seen 
otherwise.  Large  data  bases  of  information  can  be  quickly  accessed  and  traversed  in  a 
manner  that  facilitates  the  learning  of  otherwise  disparate  elements  (e.g.,  compendiums 
such  as  the  Jane's  Fighting  Ships  1985-86,  1986).  Timing  of  responses  and  the  possibility 
of  using  them  for  feedback  during  time  dependent  tasks  in  another  qualitative  feature 
(e.g.,  in  equipment  operator  tasks).  Simulations  can  represent  a  complex  and  even 
dangerous  piece  of  equipment  or  demonstrate  complex  principles  not  easily  illustrated 
with  conventional  methods  or  without  recourse  to  the  actual  equipment.  For  example,  the 
STEAMER  project  (Hollan,  Hutchins,  3c  Weitzman,  1984)  represents  a  complex  ship 
propulsion  system  in  a  manner  that  allows  students  to  rapidly  see  the  effects  of  making 
system  adjustments. 

The  decision  to  computerize  training  raises  questions  about  what  improvements  will 
be  achieved  over  existing  methods.  It  is  easy  to  foresee  improvements  resulting  from 
easing  clerical  or  administrative  tedium.  Unless  CAI  provides  some  benefit,  there  is  no 
point  in  simply  feeding  existing  lessons  into  the  computer  merely  to  automate  a  book. 
Rather  than  computerizing  entire  curricula,  a  more  rational  path  is  to  identify  selected 
CAI  applications  that  offer  an  improvement.  Some  already  cited  practical  reasons  for 
using  CAI  were  to  offer  a  learning  capability  not  possible  with  conventional  methods, 
reduce  costs  compared  to  high  fidelity  trainers,  supplement  instructor  resources, 
standardize  instruction  over  many  sites,  and  provide  onsite  individual  training. 

Another  practical  reason  in  favor  of  CBI  is  that  evaluations  of  such  courses  have 
shown  training  time  to  be  reduced  by  about  30  percent  (Orlansky  3c  String,  1981).  This 
finding  is  primarily  due  to  individualization,  since  comparisons  between  individualized  CBI 
and  individualized  conventional  instruction  found  little  time  difference.  A  recent 
systematic  review  of  previous  CBI  studies  in  secondary  education  also  shows  positive 
effects  on  student  achievement  (Bangert-Drowns,  Kulik,  3c  Kulik,  1985).  A  previous 
survey  conducted  to  determine  the  extent  of  individualized  instruction  in  445  Navy  A-  and 
C-school  courses  found  that  7  percent  are  either  self-paced  or  CMI  and  93  percent  are 
group  paced  (Micheli  3c  Ford,  1983).  Computerized  training  is  closely  associated  with 
individualized  instruction,  which  may  or  may  not  involve  self-pacing  and  controls  over 
maintaining  adequate  student  progress.  Different  implementations  of  self-paced  instruc¬ 
tion  have  experienced  varying  degrees  of  acceptance  within  the  Navy  (Thorstad,  1985). 

Montague  and  Wulfeck  (1984)  argue  that  not  all  instruction  is  appropriate  for  CAI  and 
that  improvement  of  instruction  through  CAI  will  be  a  relatively  slow  and  evolutionary 
process.  The  reasons  cited  for  this  are  that  (1)  instructional  quality  is  difficult  to  achieve 
regardless  of  the  method  of  delivery,  (2)  computers  as  instructional  tools  are  in  a 
rudimentary  state  of  development,  (3)  improvements  in  either  instructional  design  or 
computer-based  delivery  will  depend  on  fundamental  changes  in  the  scientific  base  (i.e., 
smarter  programs,  not  just  faster  hardware),  and  (4)  systematic  planning  for  acquiring, 
standardizing,  distributing,  and  incorporating  proven  instructional  programs  into  schooling 
has  not  been  done. 

Computer-based  Educational  Software  System  (CBESS);  Standardizing  CBI  Software 

Many  military  training  activities  are  buying  computers  that  are  often  not  compatible 
with  one  another.  As  a  consequence,  developing  and  sharing  common  software  packages 
requires  substantial  recoding  and  duplication  of  effort  (Dallman  et  al.,  1983).  As  a  partial 
solution,  NAVPERSRANDCEN  is  standardizing  CBI  programs  of  previous  practical  utility 
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in  Navy  training  through  a  contract  with  the  University  of  Utah.  The  computer-based 
educational  software  system  (CBESS)  is  a  set  of  C-language  programs  intended  to  allow 
greater  future  portability  as  new  computer  technology  developed  (cf.,  Brandt,  1984). 
Portability  refers  to  the  input  data  or  courseware  as  well  as  to  the  separate  computer 
language  in  which  the  programs  are  written.  Once  developed,  courseware  should  be  able 
to  be  reused  as  new  programs  are  adapted  to  other  computers.  Adaptation  of  the 
modularized  computer  program  source  code  itself  will  require  minor  changes,  but  new 
computers  may  require  rewriting  a  few  low  level  "driver"  routines,  for  new  display  devices 
for  example. 

CBESS  consists  of  four  major  elements  that  can  be  used  for  many  of  the  applications 
suggested  by  the  present  findings. 

1.  The  Equipment  Problem  Solving  Trainer  (EPST)  program  is  a  two-dimensional 
video  trainer/simulator  designed  to  reduce  reliance  on  the  use  of  actual  equipment 
trainers  in  learning  to  operate,  maintain,  and  troubleshoot  malfunctions. 

2.  The  Computer-based  Memorization  System  (CBMS)  programs  use  a  semantic 
network  to  represent  large  data  bases  of  facts  to  be  memorized  through  data-base 
browsing  and  games. 

3.  The  Language  Skills  Computer-assisted  Instruction  (LSCAI)  programs  provide 
training  in  general  and  technical  vocabulary  and  reading  through  exercises  and  games. 

4.  A  general  CBI  package  allows  creation  of  textual,  graphic,  or  video  screens. 

These  programs  generally  operate  in  an  author  mode  in  which  an  instructional  developer 
or  subject  matter  expert  enters  new  instruction  and  an  instructional  delivery  mode  in 
which  the  program  interacts  with  students. 

CBESS  is  intended  to  provide  a  systematic  way  to  implement  certain  types  of 
instruction  in  Navy  training  and  should  be  considered  a  prototype  for  future  standardiza¬ 
tion  of  relatively  portable  software  libraries.  Continuing  effort  is  needed  to  develop 
guidelines  for  determining  when  CBI  is  appropriate  and  to  develop  aids  for  CBI  authors  to 
use  to  make  quality  CBI  for  their  students.  If  a  decision  is  being  contemplated  to 
computerize  certain  aspects  of  a  course,  then  aids  to  guide  the  process  would  be  useful. 
Such  guidelines  would  address  "when  and  what"  instruction  to  computerize,  as  well  as 
"how  to"  develop  the  instruction.10  Kearsley  (1983),  for  example,  has  provided  a  general 
handbook  for  the  potential  CBI  developer.  Developing  CBI  can  take  about  200  hours  of 
development  time  for  every  hour  of  instruction  implemented.  Development  includes 
making  a  feasibility  study,  a  team  effort  encompassing  both  computer  wise  instructional 
and  technical  design  expertise,  and  evaluation  of  effectiveness.  Technical  issues  also 


10DoD  is  developing  these  guidelines:  ASD  Memorandum  for  ASA(M&RA), 

ASN(M&RA),  and  ASAF(MRA<5cI)  dated  9  September  1985;  Subject:  Guidelines  for  the 
Development,  Acquisition,  and  Use  of  Computer-based  Instruction.  The  Navy  is  also 
developing  such  guidance  (CNO  letter  Ser  1 16D/6U368509,  dated  10  March  1986;  Subject: 
Navy  Computer-based  Instruction  (CBI)  Guidelines).  Some  military  guidelines  can  also  be 
found  in  Military  Standard  MIL-STD  1472C  (Human  Engineering  Design  Criteria  for 
Military  Systems,  Equipment  and  Facilities,  1981),  but  these  do  not  specifically  address 
instructional  development. 
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arise  concerning  limitations  of  reliability  and  usability,  hardware  configuration  and 
availability,  transportability,  and  the  features  of  the  courseware  authoring  system.  The 
acquisition  of  hardware  through  competitive  procurement  may  also  be  complex.  Large 
scale  standard  contractual  arrangements  such  as  the  Air  Force/Navy  contracts  (e.g.,  for 
Tempest  certified  computers)  facilitate  procurement  for  smaller  development  efforts. 


RECOMMENDATIONS 

The  following  recommendations  are  for  the  Navy  education  and  training  community: 

1.  Computer  support  should  be  provided  for  many  administrative  and  clerical 
functions,  such  as  scheduling,  student  tracking,  and  general  record  keeping.  Student 
testing  programs  with  a  stable  high  volume  would  benefit  from  computerized  test 
administration,  scoring,  recording,  and  tracking  functions. 

2.  With  recent  initiatives  to  move  training  out  of  shore-based  schools,  training 
management,  student  record  keeping,  and  certification  testing  will  be  more  diffuse  and 
difficult  to  manage  in  the  future.  Therefore,  stand  alone-computer  tools  on  standard 
microcomputers  should  be  developed  for  shipboard  personnel  to  use  in  managing  training. 

3.  CBI  should  be  used  for  a  number  of  specific  applications.  Using  ancillary 
trainers  for  some  laboratory  situations  would  allow  better  utilization  of  student  and 
instructor  time  when  limited  access  to  high  cost  trainers  causes  excessive  waiting.  Many 
students'  entering  skills  (e.g.,  reading,  mathematics,  technical  vocabulary)  should  be 
supplemented  with  commonly  available  CBI.  Learning  objectives  involving  drill  and 
practice,  simulation,  remembering  facts,  and  use  of  procedural  steps  occur  frequently  and 
are  particularly  amenable  to  computer-based  instruction. 

4.  Continuing  work  should  be  supported  to  develop  guidelines  as  to  when  CBI  is 
appropriate  and  to  develop  aids  for  CBI  authors  so  that  they  can  develop  quality  CBI  for 
their  students. 
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APPENDIX  A 

SURVEY  QUESTIONNAIRE 


A-0 


SURVEY  QUESTIONNAIRE 


Survey  Questionnaire  As  Administered 


CIN  Number _  Course  Title 

Location  _ 


Place  a  mark  in  the  (a)  column  if  the  answers 
to  the  following  questions  are  YES.  Mark  the 
(b)  column  if  the  answers  are  NO. 


1.  Will  the  student  use  a  computer  on  the  job 
after  completing  this  course 

Do  any  of  the  following  five  areas  present 
special  problems  in  this  course? 

2.  Adequate  Learning  Objectives 

3.  Curriculum  Stability 

4.  Reading  Ability 

5.  Math  Ability 

6.  Entering  Skills 

Do  the -learning  objectives  require  any  of  the  following: 

7.  Human  Interaction 

8.  Variable  Responses 

9.  Computer  Utilization 


(a)  (b) 


(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

(b) 

(a) 

<b) 

10.  Simulation 

11.  Extensive  Drill  and  Practice 

12.  Automated  Cues  and  Prompts 


A— l 


Mark  the  (a)  column  if  the  process  is  computerized 
and  judged  adequate;  the  (b)  column  if  the  process 
is  computerized  and  judged  inadequate;  the  (c) 
column  if  the  process  is  manual  and  judged  ade¬ 
quate;  the  (d)  column  if  the  process  is  manual  and 
judged  inadequate. 


What  test  processes  are  used  in  this  course? 

13.  Performance  Evaluation 

14.  Pre-Testing 

15.  Unit/Module  Tests 

16.  Summary/Final  Tests 

17.  Remediation  Prescription 

18.  Test  Item  Bank 

19.  Test  Scoring 

20.  Test  Analysis 

21.  Recording 

Which  of  the  following  course  management  pro¬ 
cesses  are  used? 

22.  Student  Time/Test  Attempt  Records 

23.  Student  Scheduling 

24.  Resource  Scheduling 


(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 


(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 
(a)  (b)  (c)  (d) 


Mark  the  column  corresponding  to  the  state¬ 
ment  selected  below. 


25.  Which  of  the  following  statements  best  describes 
the  distribution  of  student  achievement  in  this 
course? 

(a)  a  few  students  above  average,  many  about 

average,  and  a  few  below  average.  (a)  (b)  (c) 

(b)  more  above  average  than  below  average 
students. 

(c)  more  below  average  than  above  average 
students. 
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26.  What  is  the  average  time  required  for  a  FAST  student 
to  complete  this  course? 

27.  What  is  the  average  time  required  for  a  SLOW  student 
to  complete  this  course? 

28.  How  many  modules  are  there  in  this  course? 

29.  How  many  modules  in  this  course  use  some  form 
of  computer-based  instruction? 

30.  What  computer  equipment  is  used  in  managing  or 
teaching  this  course? 

31.  What  percent  of  the  students  reach  criterion  on 
module  tests  on  the  first  attempt? 

32.  How  many  students  are  assigned  to  the  average-- 

a.  Classroom 

b.  Laboratory 

33.  How  many  instructors  are  assigned  to  the  average-- 

a.  Classroom 

b.  Laboratory 

34.  What  is  the  average  holding  time  in  terms  of  number 
of  working  days  for  students  waiting  for  this 
course  to  start? 

35.  Are  students  assigned  special  course  materials  based 
on  aptitude  scores? 

36.  If  students  are  assigned  special  materials  based  on 
aptitude  scores,  what  aptitude  measures  are  used? 
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37.  Estimate  the  percent  of  course  time  devoted  to  each 
of  the  following: 

a.  Lecture 

b.  Discussion 

c.  Demonstration 

d.  Self-study  of  reading  materials  in  class 

e.  Tutoring 

f.  Films  or  Videotapes 

g.  Laboratory 

h.  Tests 

i.  Other 

List  Other  _ 

38.  What  is  the  latest  date  for  the  following: 

a.  Project  Plan 

b.  Curriculum  Outline 

c.  Insructional  Management  Plan 

d.  Instructor  Guide 

e.  Instructional  Materials 

f.  Criterion  Referenced  Tests 

39.  What  modules  in  this  course  should  be  considered  a  high 

priority  for  some  application  of  computer  assisted 
instruction?  _ 


40.  Is  there  a  severe  student  "wait  time"  for  access  to  equipment  in  lab 

courses?  Yes _  No _ _ 

41.  If  yes,  what  modules  and  what  is  the  nature  of  the  problem? 


42.  If  computer  support  was  available  for  your  school,  what  functions  would 
have  first  priority  ?  _ _ _ 
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APPENDIX  B 
SURVEYED  COURSES 
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Surveyed  Courses 


CDPa 

CINb 

Course  Title 

Electrical  Occupational  Group 

A -School 

601B 

A 1000010 

Basic  Electricity  and  Electronics 

6274 

A1000010 

Basic  Electricity  and  Electronics 

604N 

A1000060 

dob  Oriented  Basic  Skills  (JOBS)  Basic  Skills— Electronics 

603V 

A1000062 

Strand  4 

Electronics  Technician  (ET) 

604E 

A1000064 

Electronics  Technician  Class  A— Nuclear  Field 

6146 

A1210142 

Strategic  Weapon  System  Electronic  A 

46 IX 

A1210240 

NATO  SEASPARROW  Guided  Missile  Fire  Control  System 

603H 

A1210482 

Strategic  Weapons  Systems  Electronics  Math 

6131 

A1500025 

Data  Systems  A  School 

1399 

A1500051 

USQ-20  Systems  Maintenance 

605Z 

A6230105 

Interior  Communication  Electrician  A  School 

6070 

A6620016 

Electricians  Mate  (EM) 

6219 

C0002010 

Aviation  Fundamentals  AT 

6244 

C1002010 

Advanced  First  Team  Avionics  A-l 

6241 

C1002013 

Avionics  Technician  Course  Class  A-l 

6239 

C1002013 

Avionics  Technician  Course  Class  A-l 

6315 

C6022012 

Avionics  Electronics  Mate  Class  A-l 

C-<5c-F-Schools 

350T 

A1000034 

Miniature/Microminiature  Electronic  Repair  (2M) 

016C 

A1020210 

Electronics  Warfare  Tech/AN/SLQ-32 

3427 

A1600012 

Cryptographic  Equipment  TSEC/KWR37  Maintenance 

Mechanical  Occupational  Group 

A-School 

6400 

A0410010 

Gunner's  Mate,  Missile 

6115 

A0410010 

Gunner's  Mate,  Gun 

607X 

A0410016 

Gunner's  Mate,  Missile,  Class  A  Phase  II 

607W 

A0410016 

Gunner's  Mate,  Missile,  Class  A  Phase  II 

627S 

A4950035 

Hull  Technician  A  School 

6119 

A4950035 

Hull  Technician  A  School 

6262 

A6510010 

Propulsion  Engineering  Basics 

6488 

A6510079 

Boiler  Technician  Advanced  Operator 

604M 

A6510085 

Job  Oriented  Basic  Skills  Engineering 

6487 

A6520018 

Engineman  Class  A  Course 

610P 

A6520244 

Electricial  Gas  Turbine  Systems  Technician 

cl 

CDP  =  Course  data  processing  number. 
bCIN  =  Course  identification  number. 
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CDPa 

CINb 

Course  Title 

Mechanical  Occupational  Group  (Continued) 

6501 

C6012010 

Aviation  Machinists  Mate  A-l 

6516 

C6022015 

Aviation  Structural  Mechanic  Safety  Equipment 

6517 

C6022017 

Aviation  Structural  Mechanic  Hydraulic  Course 

6459 

C6022025 

Aviation  Support  Equipment  Technician  A-l 

6518 

C6032010 

Aviation  Structural  Mechanic  A-l 

6506 

C6462010 

Aviation  Ordnance  A-l 

6391 

X777777 1 

Airman  Apprentice  Training  Course 

6396 

X777777 2 

Seaman  Apprentice 

6399 

X777777 3 

Fireman  Apprentice 

C-<5c-F  -Schools 

0015 

A495203 7 

Damage  Control  P-250  Pump  Maintenance 

8467 

A6510065 

Valve  Maintenance 

8469 

A6510067 

Pump  Maintenance 

034S 

A6510068 

Auxiliary  Machinery  Turbine  Maintenance 

8470 

A6510069 

Auxiliary  Controls,  Governors  &  Regulators  Maintenance 

Boiler  Technician  Propulsion  Plant  Maintenance 

016Y 

A6510080 

3457 

A6520019 

Basic  Engineman 

530P 

K0412048 

Magazine  Sprinkler  System  Operation,  Maintenance  &  Repair 

281B 

K4952179 

Foam  Generating  System  Operation  &  Maintenance 

Clerical/Administrative  Occupational  Group 


A-School 

6144 

A2020014 

Radioman  A  School  Common  Core 

6102 

A5000014 

Personnelman  Class  A-l  School 

528R 

A5000025 

Administration  and  Operation  of  Ships  3-M  System 

6057 

A5100012 

Yeoman  A  School 

6059 

A5510014 

Storekeeper  Class  A  School 

6477 

A8230012 

Ship's  Serviceman  Class  A  School 

0563 

A9500062 

Navy  Prior  Service  Veterans  Indoctrination  Training 

6522 

C5512010 

Aviation  Storekeeper  Course  Class  A 

6528 

C5552010 

Aviation  Maintenance  Administration  Course  Class  A-l 

6511 

X4444440 

Navy  Marine  Typing  Indoctrination 

609E 

X4444450 

Basic  Typing  Cryptological 

C-&-F  -Schools 

3516 

A012001 1 

Individualized  Learning  Supervisor,  Track  of  Instruction,  Basic 

062M 

A4950421 

Basic  Nuclear-biological-chemical  Defense 

0192 

A  4C0014 

Communications  Security,  Material  System  Custodian 

3193 

A500001 1 

Command  Career  Counselor  Course 

508U 

A5000025 

Administration  and  Operation  of  Ships  3-M  System 

2398 

A5000028 

Ships  3-M  System  Coordinator/Inspection  Team  Administration 

2949 

A5000028 

Ships  3-M  System  Coordinator/Inspection  Team  Administration 

a 

CDP  =  Course  data  processing  number. 
JCIN  =  Course  identification  number. 
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CDPa  CINbb 


Course  Title 


Clerical/Administrative  Occupational  Group  (Continued) 


C-&-F-Schools 

030T 

A5000033 

3unior  Petty  Officer  Leader,  Management  Education  Training 

029B 

A5000034 

Leader  <5c  Management  Education  Training 

061P 

A5000034 

Leader  Sc  Management  Education  Training 

2622 

A  5F0011 

Military  3ustice  -  Senior  Officer  -  Fleet 

8478 

A6510064 

Introduction  to  Engineering  System  Maintenance  Principles  Sc 
Practices 

415N 

A6520152 

Introduction  to  Engineering  System  Maintenance 

052A 

A  7C0025 

Leadership  Management  Education  Training,  Aviation  Division 
Officer 

062E 

A  7C0027 

Leader  Sc  Management  Education  Training  Warrant 

Officers/Limited  Duty  Officers 

333U 

C5512012 

Marine  Aviation  Supply  System 

079A 

32430970 

Soviet  Seapower 

2291 

32430981 

Enlisted  Intelligence  Assistant  Course  (EIAC) 

9402 

3  3A0951 

Shipboard  Intelligence  Officer  Course 

509Q 

34950400 

Damage  Control  Petty  Officer 

509L 

34950400 

Damage  Control  Petty  Officer 

509L 

34950400 

Damage  Control  Petty  Officer 

535X 

K4932009 

Unit  Safety  Supervisor 

266L 

K5002040 

Career  Information  and  Counseling 

9552 

K7C22135 

Administration  Personnel 

268U 

K8212142 

Engineering  Prouplsion  Fuels  Sc  Oil,  Shipboard  Administration 

531K 

K8302120 

Shore  Patrol  Orientation 

523G 

K8302122 

Master  at  Arms  (Afloat)  Indoctrination 

8994 

X8888880 

Specialized  Briefing  Training  Senior  Division  Officers 

Operator  Occupational  Group 


A-School 

604T 

A1000059 

30BS  Operator  Course  -  Stand  2 

6083 

A1020209 

Electronic  Warfare  Operations 

602C 

A1020209 

Electronic  Warfare  Operations 

6015 

A1300037 

Surface  Antisubmarine  Warfare  (ASW)  Operator 

6540 

A221001 1 

Operations  Specialist  A 

6301 

A2310044 

Cryptologic  Technician  R 

6302 

A2310045 

Non-Morse  Basic  Preparatory  Operations 

6320 

A2310046 

AN/GSO-76  (TEBO)  Operations 

601E 

A2600030 

Cryptologic  Technician  D 

6065 

A4500010 

Music  Basic 

6167 

A5310016 

Data  Processing  Technician  A  School 

6125 

A8000013 

Mess  Management  Specialist  A  Course 

6537 

C2102010 

Aviation  Antisubmarine  Warfare  Operator  School  A-l 

6278 

C2222010 

Air  Traffic  Control 

aCDP  =  Course  data  processing  number. 
bCIN  =  Course  identification  number. 
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CDPa 

CINb 

Course  Title 

Operator  Occupational  Group  (Continued) 

C-5c-F -Schools 

3934 

A0600025 

Underway  Replenishment  Operator 

5382 

A2010022 

Morse  Code  Operator 

9320 

A  4H0118 

Surface  Warfare  Officer  Basic 

5066 

30410140 

Small  Arms,  Expert  Rifle 

5184 

30600036 

Boatswain  Training 

274E 

30 620634 

Basic  Boat  Coxswain 

2883 

3  2G0604 

Rules  of  the  Nautical  Road 

5292 

34950413 

Shipboard  Aircraft  Fire  Fighting  Training 

530B 

K0602023 

Pilot  Rescue  and  Associated  Water  Survival 

9334 

K  2E2108 

Safe  Shiphandling 

080K 

K4952184 

Foam  Generating  Systems  Engineering  Operator  Personnel 

274T 

K6522157 

Sewage  Disposal  System  Operation  and  Maintenance 

Team  Training 

C-ic-F -Schools 

075T 

C2222017 

CV  Carrier  Air  Traffic  Controller  Team  Training 

8714 

31130170 

NAV  Gunfire  Support  MK  68  Team  Training 

2493 

32100513 

Surface  Ship  ASW  Attack  Team  Training 

538P 

32210310 

CIC  Team  Training 

8747 

32210345 

RADAR  Naval  Team  Training 

538Q 

32210357 

Multi-threat 

9305 

3  2G0530 

ASW  Task  Group  Course 

533E 

3  2G0531 

ASW  Surface  Unit  Course 

512Y 

34950412 

General  Shipboard  Fire  Fighting  Training 

509X 

34950412 

General  Shipboard  Fire  Fighting  Training 

509Z 

34950413 

Shipboard  Aircraft  Fire  Fighting  Training 

4555 

34590414 

Air  Capable  Ship  Helicopter  Firefighting  Team  Training 

274N 

34590414 

Air  Capable  Ship  Helicopter  Firefighting  Team  Training 

4554 

34950418 

Shipboard  Fire  Fighting  Team  Training 

5 10B 

34950418 

Shipboard  Fire  Fighting  Team  Training 

5313 

K06021 19 

Underway  Replenishment  Simulator 

204L 

K0602136 

Pilot  Rescue  Team  Training 

Other  Training 

A -School 

609F 

A3333330 

Reading  Remediation 

601M 

A9500061 

Academic  Remedial  Training 

C-&-F  -Schools 

202C 

30000003 

Overseas  Homeported  Ship  Personnel  Basic 

CDP  =  Course  data  processing  number. 
bCIN  =  Course  identification  number. 
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APPENDIX  C 

COURSES  REPORTING  THE  USE  OF  COMPUTER-BASED  INSTRUCTION 
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Courses  Reporting  the  Use  of  Computer-based  Instruction 


CDPa 

CINt> 

Course  Title 

No.  of 
Modules 
or  Lessons 

016C 

A1020210 

Electronics  Warfare  Technician/AN/SLW-32 

20 

075T 

C2222017 

CV  Carrier  Air  Traffic  Controller  Team  Training 

5 

601E 

A2600030 

Cryptologic  Technician  D 

7 

601M 

A9500061 

Academic  Remedial  Training 

9 

603V 

A1000062 

Electronics  Technician  (ET) 

12 

604E 

A 1000064 

Electronics  Technician  Class  A,  Nuclear  Field 

11 

6057 

A5100012 

Yeoman  A  School 

12 

6080 

A 1020209 

Electronic  Warfare  Operations 

31 

6102 

A5000014 

Personnelman  Class  A-l  School 

1 

6219 

C00020 10 

Aviation  Fundamentals  AT 

3 

6239 

C1002013 

Avionics  Technician  Course  Class  A-l 

5 

6241 

C1002013 

Avionics  Technician  Course  Class  A-l 

5 

6301 

A2310044 

Cryptological  Technician  R 

1 

6487 

A6520018 

Engineman  Class  A  Course 

16 

6540 

A2210011 

Operations  Specialist  A 

260 

8747 

02210345 

RADAR  Naval  Team  Training 

5 

266L 

K5002040 

Career  Information  and  Counseling 

2 

aCDP  =  Course  data  processing  number. 
bCIN  =  Course  identification  number. 
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APPENIDX  D 

SATISFACTION  WITH  EXISTING  TEST  METHODS  AND 
COURSE  MANAGEMENT  PROCESSES:  ALTERNATE  PRESENTATIONS 
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Table  D-l 


Test  Methods  Employed:  Satisfaction  and 
Dissatisfaction  Recalculated 


Test  Methods  Used 

Test  Method  Employed  (%) 

Computer 

Manual 

OK 

NO 

OK 

NO 

Performance  evalution 

75.0 

25.0 

82.6 

17.3 

Pretesting 

80.0 

20.0 

53.6 

46.3 

Unit/module  tests 

88.2 

11.7 

72.0 

28.0 

Summary/final  tests 

90.9 

9.1 

68.8 

31.2 

Remediation  prescription 

83.3 

16.6 

66.6 

33.3 

Test  item  bank 

77.7 

22.2 

68.7 

31.3 

Test  scroing 

75.0 

25.0 

61.3 

38.7 

Test  analysis 

64.5 

35.5 

55.1 

44.9 

Recording 

81.5 

18.5 

58.1 

41.9 

Notes. 


1.  Percentages  obtained  by  dividing  the  Computer  OK  &  NO 
response  percentages  in  Table  7  by  their  sum.  The  same  computation 
was  performed  separately  for  the  Manual  OK  <5c  NO  response 
percentages  in  Table  7.  These  percentages  were  calculated  so  that 
readers  could  compare  OK  and  NO  response  percentages  for  just 
computer  or  just  manual  method  users. 

2.  Percentages  do  not  always  total  100  due  to  rounding. 

3.  N  =  135. 
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Table  D-2 


Course  Management  Processes: 
Satisfaction  and  Dissatisfaction  Recalculated 


Management  Process  Used 

Responses  (%) 

Computer 

Manual 

OK 

NO 

OK 

NO 

Student  time/test- 

attempt  records 

70.0 

30.0 

66.2 

33.8 

Student  scheduling 

70.9 

29.1 

75.9 

24.1 

Resource  scheduling 

60.0 

40.0 

72.8 

27.2 

Notes. 

1.  Percentages  obtained  by  dividing  the  Computer  OK  &  NO 
response  percentages  in  Table  11  by  their  sum.  The  same 
computation  was  performed  separately  for  the  Manual  OK  6c  NO 
response  percentages  in  Table  11.  These  percentages  were  calculated 
so  that  readers  could  compare  OK  and  NO  response  percentages  for 
just  computer  or  just  manual  method  users. 

2.  N  =  135. 
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Military  Assistant  for  Training  and  Personnel  Technology  (OUSD)  (R&AT) 
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Commander,  Training  Command,  U.S.  Pacific  Fleet 

Technical  Director,  U.S.  ARI,  Behavioral  and  Social  Sciences,  Alexandria,  VA  (PERI-ZT) 
Commander,  OPSTNGDIV  Air  Force  Human  Resources  Laboratory,  Williams  Air  Force 
Base,  AZ  (AFHRL/OT) 

Director,  Training  Systems  Development,  Randolph  Air  Force  Base,  TX  (Hq  ATC/XPRS) 
Superintendent,  Naval  Postgraduate  School 
Director  of  Research,  U.S.  Naval  Academy 
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Dr.  Ackles,  Canadian  Defense  Liaison  Staff,  Washington,  DC 
Canadian  Forces  Personnel,  Applied  Research  Unit,  Canada 
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Ministry  of  Defense,  Senior  Psychologist,  Naval,  England 

D.  Dennison,  Army  Personnel  Research  and  Establishment,  Personnel  Psychological 
Division,  England  (2) 
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